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SECTION  1.  GENERAL  INFORMATION 


1.1  BACKGROUND 

Technologies  under  development  for  the  detection  and  discrimination  of  military 
munitions  (MM)  (i.e.,  unexploded  ordnance  {UXO}  and  discarded  military  munitions  {DMM}) 
require  testing  so  that  performance  can  be  characterized.  To  that  end,  Standardized  Test  Sites 
have  been  developed  at  Aberdeen  Proving  Ground  (APG),  Maryland,  and  U.S.  Army  Yuma 
Proving  Ground  (YPG),  Arizona.  These  test  sites  provide  a  diversity  of  geology,  climate, 
terrain,  and  weather  as  well  as  diversity  in  munitions  and  clutter.  Testing  at  these  sites  is 
independently  administered  and  analyzed  by  the  government  for  the  purposes  of  characterizing 
technologies,  tracking  perfonnance  with  system  development,  comparing  performance  of 
different  systems,  and  comparing  performance  in  different  environments. 

The  Standardized  UXO  Technology  Demonstration  Site  Program  is  a  multiagency 
program  spearheaded  by  the  U.S.  Army  Environmental  Command  (USAEC).  The  U.S.  Army 
Aberdeen  Test  Center  (ATC)  and  the  U.S.  Army  Corps  of  Engineers  Engineering  Research  and 
Development  Center  (ERDC)  provide  programmatic  support.  The  program  is  being  funded  and 
supported  by  the  Environmental  Security  Technology  Certification  Program  (ESTCP),  the 
Strategic  Environmental  Research  and  Development  Program  (SERDP),  and  the  Army 
Environmental  Quality  Technology  (EQT)  Program. 

1.2  SCORING  OBJECTIVES 

The  objective  in  the  Standardized  UXO  Technology  Demonstration  Site  Program  is  to 
evaluate  the  detection  and  discrimination  capabilities  of  a  given  technology  under  various  field 
and  soil  conditions.  Inert  munitions  and  clutter  items  are  positioned  in  various  orientations  and 
depths  in  the  ground. 

The  evaluation  objectives  are  as  follows: 

a.  To  determine  detection  and  discrimination  effectiveness  under  realistic  scenarios  with 
various  targets,  geology,  clutter,  density,  topography,  and  vegetation. 

b.  To  determine  cost,  time,  and  workforce  requirements  to  operate  the  technology. 

c.  To  determine  demonstrator’s  ability  to  analyze  survey  data  in  a  timely  manner  and 
provide  prioritized  Target  Lists  with  associated  confidence  levels. 

d.  To  provide  independent  site  management  to  enable  the  collection  of  high  quality, 
ground-truth,  geo-referenced  data  for  post-demonstration  analysis. 
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1.2.1  Scoring  Methodology 

a.  The  scoring  of  the  demonstrator’s  performance  is  conducted  in  two  stages:  response 
stage  and  discrimination  stage.  For  both  stages,  the  probability  of  detection  (Pd)  and  the  false 
alarms  are  reported  as  receiver-operating  characteristic  (ROC)  curves.  False  alarms  are  divided 
into  those  anomalies  that  correspond  to  emplaced  clutter  items,  measuring  the  probability  of 
clutter  detection  (Pcd)  or  the  probability  of  false  positive  (Pfp).  Those  that  do  not  correspond  to 
any  known  item  are  tenned  background  alarms.  The  background  alanns  are  addressed  as  either 
probability  of  background  alarm  (Pba)  or  background  alarm  rate  (BAR). 

b.  The  response  stage  scoring  evaluates  the  ability  of  the  system  to  detect  emplaced 
targets  without  regard  to  ability  to  discriminate  munitions  from  other  anomaly  sources.  For  the 
blind  grid  response  stage,  the  demonstrator  provides  a  target  response  from  each  and  every  grid 
square  along  with  a  threshold  below  which  target  responses  are  deemed  insufficient  to  warrant 
further  investigation.  This  list  is  generated  with  minimal  processing  and,  since  a  value  is 
provided  for  every  grid  square,  includes  amplitudes  both  above  and  below  the  system  noise  level. 
For  the  open  field,  the  demonstrator  provides  a  list  of  all  anomalies  deemed  to  exceed  a 
demonstrator  selected  target  detection  threshold.  An  item  (either  munition  or  clutter)  is  counted 
as  detected  if  a  demonstrator  indicates  an  anomaly  within  a  specified  distance  (Halo  Radius 
(Rhaio))  of  a  ground  truth  item. 

c.  The  discrimination  stage  evaluates  the  demonstrator’s  ability  to  correctly  identify 
munitions  as  such  and  to  reject  clutter.  For  the  blind  grid  discrimination  stage,  the  demonstrator 
provides  the  output  of  the  discrimination  stage  processing  for  each  grid  square.  For  the  open 
field,  the  demonstrator  provides  the  output  of  the  discrimination  stage  processing  for  anomaly 
reported  in  the  response  stage.  The  values  in  these  lists  are  prioritized  based  on  the 
demonstrator’s  determination  that  a  location  is  likely  to  contain  munitions.  Thus,  higher  output 
values  are  indicative  of  higher  confidence  that  a  munitions  item  is  present  at  the  specified 
location.  For  digital  signal  processing,  priority  ranking  is  based  on  algorithm  output.  For  other 
discrimination  approaches,  priority  ranking  may  be  based  on  rule  sets  or  human  judgment.  The 
demonstrator  also  specifies  the  threshold  in  the  prioritized  ranking  that  provides  optimum 
performance,  (i.e.  that  is  expected  to  retain  all  detected  munitions  and  reject  the  maximum 
amount  of  clutter). 

d.  The  demonstrator  is  also  scored  on  efficiency  and  rejection  ratios,  which  measure  the 
effectiveness  of  the  discrimination  stage  processing.  The  goal  of  discrimination  is  to  retain  the 
greatest  number  of  munitions  detections  from  the  anomaly  list,  while  rejecting  the  maximum 
number  of  anomalies  arising  from  nonmunitions  items.  Efficiency  measures  the  fraction  of 
detected  munitions  retained  after  discrimination,  while  the  rejection  ratio  measures  the  fraction 
of  false  alanns  rejected.  Both  measures  are  defined  relative  to  the  maximum  number  of 
munitions  detectable  by  the  sensor  and  its  accompanying  clutter  detection/false  positive  rate  or 
BAR. 
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e.  Based  on  configuration  of  the  ground  truth  (GT)  at  the  standardized  sites  and  the 
defined  scoring  methodology,  in  some  cases,  there  exists  the  possibility  of  having  anomalies 
within  overlapping  halos  and/or  multiple  anomalies  within  halos.  In  these  cases,  the  following 
scoring  logic  is  implemented: 

(1)  In  situations  where  multiple  anomalies  exist  within  a  single  Rhaio,  the  anomaly  with 
the  strongest  response  or  highest  ranking  will  be  assigned  to  that  particular  GT  item.  If  the 
responses  or  rankings  are  equal,  then  the  anomaly  closest  to  the  GT  item  will  be  assigned  to  the 
GT  item.  Remaining  anomalies  are  retained  and  scored  until  all  matching  is  complete. 

(2)  Anomalies  located  within  any  Rhai0  that  do  not  get  associated  with  a  particular  GT 
item  are  excess  alanns  and  will  be  disregarded. 

f.  In  some  cases,  groups  of  closely  spaced  munitions  have  overlapping  halos.  The 
following  scoring  logic  is  implemented  (fig.  A-l  through  A-9): 

(1)  Overall  site  scores  (i.e.,  Pd)  will  consider  only  isolated  munitions  and  clutter  items. 

(2)  GT  items  that  have  overlapping  halos  (both  munitions  and  clutter)  will  form  a  group 
and  groups  may  form  chains. 

(3)  Groups  will  have  a  complex  halos  composed  of  the  composite  halos  of  all  its  GT 

items. 

(4)  Groups  will  have  three  scoring  factors:  groups  found,  groups  identified,  and  group 
coverage.  Scores  will  be  based  on  1 : 1  matches  of  anomalies  and  GT. 

(a)  Groups  Found  (Found):  the  number  of  groups  that  have  one  or  more  GT 
items  matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with 
detecting  a  group  if  any  item  within  the  group  is  matched  to  an  anomaly  in  their  lists. 

(b)  Groups  Identified  (ID):  the  number  of  groups  that  have  two  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  identifying 
that  a  group  is  present  if  multiple  items  within  the  composite  halo  are  matched  to  anomalies  in 
their  lists. 

(c)  Group  Coverage  (Coverage):  the  number  of  GT  items  matched  within  groups  divided 
by  the  total  number  of  GT  items  within  groups.  This  metric  measures  the  demonstrator  accuracy 
in  determining  the  number  of  anomalies  within  a  group.  If  five  items  are  present  and  only  two 
anomalies  are  matched,  the  demonstrator  will  score  0.4.  If  all  five  are  matched,  the  demonstrator 
will  score  1.0. 

(5)  Location  error  will  not  be  reported  for  groups. 
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(6)  Demonstrators  will  not  be  asked  to  call  out  groups  in  their  scoring  submissions.  If 
multiple  anomalies  are  indicated  in  a  small  area,  the  demonstrator  will  report  all  individual 
anomalies. 

(7)  Excess  alarms  within  a  halo  will  be  disregarded. 

g.  All  scoring  factors  are  generated  utilizing  the  Standardized  UXO  Probability  and 
Plot  Program,  version  4. 

1.2.2  Scoring  Factors 

Factors  to  be  measured  and  evaluated  as  part  of  this  demonstration  include: 

a.  Response  stage  ROC  curves: 

(1)  Probability  of  detection  (Pdres). 

(2)  Probability  of  clutter  detection  (Pcd). 

(3)  Background  alann  rate  (BARres)  or  probability  of  background  alarm  (Pb/es). 

b.  Discrimination  stage  ROC  curves: 

(1)  Probability  of  detection  (Pddlsc). 

(2)  Probability  of  false  positive  (Pfp). 

(3)  Background  alann  rate  (BARdlsc)  or  probability  of  background  alarm  (Pbadlsc). 

c.  Metrics: 

(1)  Efficiency  (E). 

(2)  False  positive  rejection  rate  (Rfp). 

(3)  Background  alarm  rejection  rate  (Rba). 

d.  Other: 

(1)  Probability  of  detection  by  size,  depth,  and  density. 

(2)  Classification  by  type  (i.e.,  20-,  40-,  105mm,  etc.). 

(3)  Location  accuracy  for  single  munitions. 

(4)  Equipment  setup,  calibration  time,  and  conesponding  worker-hour  requirements. 

(5)  Survey  time  and  corresponding  worker-hour  requirements. 
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(6)  Reacquisition/resurvey  time  and  worker-hour  requirements  (if  any). 

(7)  Downtime  due  to  system  malfunctions  and  maintenance  requirements. 


5 


(Page  6  Blank) 


SECTION  2.  DEMONSTRATION 


2.1  DEMONSTRATOR  INFORMATION 

2.1.1  Demonstrator  Point  of  Contact  (POC)  and  Address 

POC:  Mr.  Skip  Snyder 

408  954-0522 

Address:  G&G  Sciences,  Inc. 

23  Road 

Grand  Junction,  CO  81505 

2.1.2  System  Description  (provided  by  demonstrator) 

With  support  from  the  Environmental  Security  Certification  Program  (ESTCP)  Geometries 
is  commercializing  an  advanced  electromagnetic  induction  (EMI)  system  for  UXO  detection  and 
characterization.  The  system  will  have  dual-mode  (electromagnetic/magnotometer  (EM/MAG)) 
capability.  Called  the  Metal  Mapper,  the  new  system  draws  elements  of  its  design  from 
advanced  systems  currently  being  developed  by  G&G  Sciences,  Inc.  with  funding  from  Naval 
Systems  Command  (NAVSEA)  (the  advanced  ordnance  locator  (AOL)  2  project)  and  by  the 
Lawrence  Berkeley  Laboratories  (the  Berkely  UXO  discriminator  (BUD)  system)  with  SERDP 
and  ESTCP  funding.  The  Metal  Mapper  system  is  unique  and  innovative  in  several  respects: 

a.  Multiple  Transmitter  Loops  1:  The  Metal  Mapper  antenna  platform  includes  three 
mutually  orthogonal  transmitter  loops. 

b.  3-Axis  Sensor  Array2:  The  Metal  Mapper  antenna  platfonn  includes  a  spatial  array  of 
seven  3-axis  receiver  antennas  (21  independent  measurements  of  the  secondary  magnetic  field). 

c.  Electronically  Switched  time-gated  electromagnetic  (TEM)  Transmitter  Loop 
Driver:  The  Metal  Mapper  system  is  unique  in  its  ability  to  drive  its  transmitter  loop  array. 
Under  control  of  the  data  acquisition  (DAQ)  computer,  the  output  of  the  transmitter  can  be 
directed  to  any  single  loop  or  automatically  multiplexed  between  loops.  There  is  also  control  of 
the  fundamental  wavefonn  period,  duty-cycle,  and  pulse  polarity.  Typically,  however,  the  loops 
are  driven  with  a  classical  bipolar  pulse  type  TEM  waveform  (i.e.,  alternating  pulse  polarity  with 
a  50-percent  duty-cycle.  Depending  on  the  survey  mode  (e.g.,  Static/Dynamic),  the  fundamental 
frequency  of  transmission  can  be  varied  from  a  low  of  1 . 1  !</<$  10  Hz. 
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Figure  1.  Demonstrator’s  system,  Metal  Mapper. 


2.1.3  Data  Processing  Description  (provided  by  demonstrator) 

Acquisition  Modes. 

a.  The  Metal  Mapper  system  is,  by  design,  a  very  flexible  system  for  acquisition  of  time 
domain  EM  (TEM)  data.  It  is  beyond  the  scope  of  this  document  to  fully  describe  that 
flexibility.  Simply  stated,  data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of 
transmitter  cycle  Repeats.  Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable 
and  are  varied  in  multiplicative  factors  of  three.  It  has  two  data  acquisition  modes: 

(1)  Static-Mode  Acquisition.  In  this  mode,  data  sampled  transients  from  each  of  the 
2 1  receiver  loops  plus  a  channel  to  sense  the  transmitter  loop  current  are  rectified  and  stacked  for 
a  specified  number  of  acquisition  blocks.  The  resulting  transients  are  (optionally)  decimated  into 
a  set  of  logarithmically  spaced  time  gates  after  which  they  are  stored  to  a  single  binary  data  file. 
As  its  name  implies,  static-mode  acquisition  is  used  to  obtain  precise  data  while  the  antenna 
platform  is  parked  at  a  single  spatial  data  point. 

(2)  Continuous-Mode  Acquisition.  As  its  name  implies,  continuous-mode  data 
acquisition  results  in  the  data  acquisition  cycle  being  repeated  until  the  operator  intervenes  to 
halt  it.  Each  of  the  Data  Points  are  appended  to  single  binary  data  file  and  thus  the  resulting  data 
file  may  consists  of  10s  or  even  100s  of  data  points.  This  mode  is  used  for  dynamic  surveying. 
Typically,  a  data  file  consists  of  all  the  points  acquired  along  a  single  profile. 
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b.  Regardless  of  the  acquisition  mode,  the  TEM  data  thus  acquired  includes  the  most 
current  Global  Positioning  System  (GPS)  position  and  the  platform  attitude  angles  (magnetic 
heading,  pitch,  and  roll).  By  properly  selecting  the  block  period  (T)  and  the  repeat  factor  (R),  the 
operator  can  set  the  sampling  rate  to  as  high  30  samples/sec.  As  we  have  stated  above,  the  data 
are  stored  as  binary  formatted  files.  However,  the  processing  software  includes  the  capability  to 
export  the  data  to  a  Geosoft  Oasis  Montaj  data  base  for  further  quality  control  (QC)  and  map 
compilation.  The  processing  also  includes  the  capability  to  export  the  data  to  text  files  and  to 
Matlab™. 

c.  Target  Selection.  G&G  Sciences,  Inc.  plan  is  to  complete  dynamic  surveys  over  both 
the  calibration  and  blind  grids.  The  surveys  will  consist  of  parallel  profiles  acquired  with 
1 -meter  offsets.  Using  these  data,  we  will  compile  a  detection  parameter  map  of  the  surveyed 
area.  The  detection  map  is  based  on  the  magnitude  of  the  secondary  fields  measured  at  each  of 
the  nine  triaxial  receiver  sensors.  The  following  processing  steps,  accomplished  using  Geosoft 
Oasis  montaj™,  are  required. 

Target  Selection. 

a.  The  plan  is  to  complete  dynamic  surveys  over  the  calibration  grid,  the  blind  grid,  the 
indirect- fire  area,  and  the  direct- fire  area.  These  surveys  will  consist  of  parallel  profiles  acquired 
with  1 -meter  offsets.  From  these  data,  we  will  compile  detection  maps  and  associated  target  lists 
for  each  of  the  areas  surveyed.  The  detection  map  is  based  on  the  magnitude  of  the  secondary 
fields  measured  at  each  of  the  seven  triaxial  receiver  sensors.  The  following  processing  steps, 
accomplished  using  Geosoft  Oasis  montaj™,  are  required: 

b.  Metal  Mapper  data  are  recorded  as  binary  files.  These  data  are  imported  directly  into 
an  OM  data  base  where  simple  editing  (e.g.,  editing  line  numbers,  deselecting  duplicate  lines, 
trimming  and  deleting  bad  data  or  stops,  etc).  All  other  steps  are  accomplished  from  within  OM 
using  its  standard  editing  and  processing  capabilities  supplemented  where  necessary  with  custom 
Geosoft  Executables  (GXs)  and  Geosoft  Scripts  (GSs)  and  Geosoft  mathematical  expression 
(EXP)  files. 

c.  Convert  Lat/Lon  to  UTM  coordinates. 

d.  Compute  detector  gate  values  for  each  of  the  21  receiver  channels. 

e.  Normalize  detector  gate  values  by  transmitter  current. 

f.  Select  background  and  remove  background  (leveling). 

g.  Generate  vector  magnitude  channels  for  each  of  seven  triaxial  receiver  cubes. 

h.  Make  heading  channel  for  each  profile. 

i.  Split  each  profile  into  seven  separate  profiles,  corrected  for  heading  and  offset  distance 
from  the  platfonn  measure  point  (generates  seven  parallel  profiles  with  13-cm  offsets). 
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j.  Grid  cube  amplitude  data. 

k.  Apply  grid  smoothing  filters  if  necessary. 

l.  Select  targets  using  an  amplitude  threshold.  The  (tunable)  parameters  are: 

(1)  Signal  amplitude. 

(2)  Detector  gate  (step  3). 

m.  Edit  target  list  based  on  inspection  of  profiles. 

Target  Re- Acquisition  and  Parameter  Estimation. 

a.  Each  of  the  targets  generated  from  the  detection  map  created  from  the  dynamic  data  are 
reacquired  with  the  Metal  Mapper  using  a  combination  of  GPS  to  return  to  the  approximate 
target  location  and  then  a  real-time  graphics  display  that  allows  the  operator  to  center  the  antenna 
platform  directly  over  the  target.  Once  the  target  has  been  reacquired,  a  static  data  set  is 
acquired  at  that  position.  In  its  static  acquisition  mode,  all  three  transmitter  loops  are  energized 
in  turn.  Typically,  a  static  data  set  will  consist  of  a  stack  of  50  to  100  data  blocks  and  the 
acquisition  parameters  are  selected  so  that  8.33  ms  or  25  ms  transients  are  acquired.  These  data 
are  recorded  in  the  same  standard  binary  format  as  is  the  dynamic  data.  However,  each  data  file 
includes  only  a  single  (stacked)  data  point  rather  than  a  sequence  of  data  points  that  are  stored  in 
a  data  file  recorded  in  the  continuous  acquisition  mode. 

b.  Each  of  the  static  data  files  are  used  as  input  to  the  Metal  Mapper  Inversion 
(MM/RMP)  program. 

c.  The  MM/RMP  program  is  a  physics-based  inversion  program  based  on  approximating 
the  transient  response  of  compact  metallic  objects  with  a  point  dipole  characterized  by  a  time6 
varying  anisotropic  polarizability  tensor.  MM/RMP  is  actually  a  wrapper  for  an  implementation 
of  the  inverse  dipole  modeling  problem  developed  by  Torquil  Smith  at  LBL  in  connection  with 
the  BUD  development  project  [1],  The  program  provides  optimum  estimates  the  following 
parameters: 

(1)  Target  position  (x,  y,  and  z):  The  3-dimensional  position  of  the  target  with  respect  to 
the  position  of  the  antenna  platfonn.  The  Metal  Mapper  includes  an  apparatus  that  senses  the 
platform  attitude  angles  (heading,  pitch,  and  roll).  Thus  the  target  position  relative  to  the 
platform  coordinate  system  can  be  converted  to  geographic  coordinates. 

(2)  Target  attitude  (heading,  pitch,  and  roll):  The  Metal  Mapper  Inversion  software 
estimates  the  target  attitude  by  finding  the  principal  coordinate  system  for  the  target 
polarizability. 
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(3)  Principal  polarizability  transients  (PI,  P2,  and  P3):  The  Metal  Mapper  Inversion 
software  estimates  the  three  principal  polarizability  transients  for  the  target.  Examples  of  the 
polarizability  curves  estimated  by  three  different  programs  using  a  static  data  set  collected  with 
the  AOL  at  YPG  last  year.  The  nine  parameters  enumerated  previously  together  with  the 
inversion  fit  statistics  are  the  fundamental  data  derived  from  the  Metal  Mapper  inversion.  In 
particular,  the  principal  polarizability  transients  those  containing  information  about  the  target. 
For  example,  if  both  targets  are  elongated  and  exhibit  a  single  axis  of  symmetry  as  indicated  by 
the  fact  that  there  is  a  single  major  polarizability  transient  and  two  nearly  identical  minor 
polarizability  curves.  A  measure  of  target  size  is  provided  by  the  integration  beneath  the 
polarizability  curves.  Note  that  the  units  of  the  polarizability  (rate)  transients  are  m  /s,  or, 
equivalently  cnrVps.  When  integrated  over  time  to  find  the  area  beneath  the  curve,  the  results 
are  units  of  volume  (m  or  cm  )  as  shown  in  the  formula  below: 

p0  =  p{t=Q,)  =  l^ldt 
*  dt 

d.  G&G  Sciences,  Inc.  uses  the  root  mean  square  (RMS)  value  of  the  three  Pqs  that  can 
calculate  from  the  three  principal  polarizability  transients  that  characterized  each  of  the  targets  as 
an  indication  of  size.  The  parameter  Pq  defined  in  equation  is  an  example  of  a  so-called 
metaparameter  that  can  be  derived  from  the  more  fundamental  target  data  that  are  the  three 
principal  polarizability  curves.  For  simple  classification  by  shape,  one  can  define  other 
meta-parameters  involving  the  relationship  of  the  three  integrated  polarizability  parameters  (Pox, 
P Oy,  and  P(h)  derived  from  equation  to  identify  elongate  targets  with  an  axis  of  symmetry.  Such 
target  features  have  been  used  effectively  by  many  to  develop  classification  metrics  [2,  3]. 
Among  the  more  useful  parameters  are  the  following: 

(1)  Transverse  Polarizability:  Pot  =  ( Poy  +  TVfy/2 

(2)  Polarizability  Ratio:  Rpo  =  Pox/Pot 

(3)  Eccentricity:  EPO  =  \P0y  -  PozVPox 

e.  Generally  speaking,  UXO  have  a  polarizability  ratio  RPo  >  1  and  an  eccentricity 
EPo«  1  indicating  an  elongate  body  with  an  axis  of  symmetry.  The  thresholds  of 
discrimination  for  a  classifier  are  detennined  using  a  set  of  training  parameters  derived  from  a 
data  set  for  which  the  ground  truth  is  known  (e.g.,  the  calibration  lanes). 

f.  Using  the  training  data,  a  classifier  based  on  principles  of  pattern  recognition  using  the 
two  or  three  most  significant  parameters  was  developed.  Typically  the  classifier  is  based  on  the 
searching  of  the  nearest  neighbors  in  order  to  find  the  (binary)  decision  boundary  providing  the 
best  division  between  ordnance  (O)  and  clutter  (C).  To  facilitate  the  development  of  a  classifier 
for  a  particular  data  set,  we  use  the  Duke  Pattern  Recognition  Toolbox  (DPRT),  a  library  of 
MatFab  functions  for  pattern  recognition  developed  by  Feslie  Collins  and  her  colleagues  at  Duke 
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University.  DPRT  supports  the  development  of  a  variety  of  classifiers  including  kNN 
(‘k’  nearest  neighbors)  and  FLD  (Fisher  Linear  Discriminant).  In  our  limited  experience,  the 
kNN  classifier  (with  k  =  3)  does  better  than  the  FLD  classifier  and  the  two.  The  two  parameters 
are  the  eccentricity  ( E)  and  the  polarizability  ratio  (R).  The  results  from  the  kNN  classifier  are 
effective  at  discriminating  between  loops  and  other  targets  with  good  symmetry.  However,  there 
is  no  basis  from  this  data  set  to  discriminate  the  shot  puts  from  other  targets.  Indeed,  the  AOL2 
polarizability  results  show  that  a  number  of  target  types  such  as  the  M75,  MK1 18  Rockeye,  and 
BLU-26  exhibit  three  nearly  identical  principal  polarizability  curves,  thus  indicating  near 
isotropic  polarizability.  However,  the  shapes  of  the  principal  polarizabilities  for  each  of  the 
targets  are  distinctly  different. 

g.  Training.  The  performance  of  the  classifier  is  very  much  dependent  both  on  the  quality 
of  the  training  data  set  and,  as  well,  on  the  choice  of  the  relevant  parameters  used  in  training. 
Scores  from  both  ATC  and  the  Institute  for  Defense  Analysis  (IDA),  based  on  the  target  list  that 
was  submitted  last  year  for  AOL  work,  indicated  that  there  were  positive  results  on  both 
detection  (Pd  =  95  percent)  and  on  discrimination  (Pd  =  85  percent).  But  the  training  data  were 
flawed  in  the  sense  that  none  of  the  targets  in  the  calibration  lanes  is  truly  clutter.  In  that  regard, 
it  is  noted  that  the  newly  reconfigured  calibration  grid  at  APG  now  includes  eight  items  of  true 
clutter  and  with  this  next  opportunity  to  submit  a  classified  target  list,  it  can  hopefully  improve 
the  discrimination  stage  scores. 

Parameter  Estimation. 

a.  Which  characteristics  will  be  extracted  from  each  detected  item  and  input  to  the 
discrimination  algorithm  (e.g.,  depth,  size,  polarizability  coefficients,  fit  quality,  etc.)? 

The  following  parameters  represent  the  subset  of  parameters  extracted  from  each  static 
data  set  acquired  over  a  detected  target.  These  parameters  were  used  in  the  generation  of  a  scalar 
metric  used  as  the  basis  for  discrimination. 

Anomaly  signal-to-noise  ratio  (SNR):  This  scalar  parameter  represents  the  ratio  of  the 
static  anomaly  amplitude  (after  background  subtraction)  measured  over  the  target  divided  by  a 
similarly  calculated  value  representing  in  effect  the  RMS  background  noise. 

Fit  statistic:  This  scalar  is  a  measure  of  the  quality  of  the  dipole  fit  to  the  observed 
static  data. 

Target  position  (X,  Y,  and  Z):  This  vector  represents  the  vector  offset,  expressed  in 
geographic  coordinates  from  the  platform  reference  point  (center  of  the  Z  transmitter  coil)  to  the 
target  center.  When  added  to  the  geographic  position  of  the  platfonn  reference  point,  these 
parameters  indicate  the  geographic  position  of  the  target. 
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Target  attitude  (H,P,R):  These  are  three  angles  representing  the  heading  (H),  pitch  (P), 
and  roll  (R)  angles  between  the  principal  axis  coordinate  system  of  the  target  (i.e.,  x-axis  =  axis 
of  symmetry)  and  the  geographic  coordinate  system  (positive  x-axis  is  grid  east,  positive  y-axis 
is  grid  north,  positive  z-axis  is  up).  Heading  is  measured  positive  clockwise  with  respect  to  grid 
north  (Y).  Pitch  is  measured  with  respect  to  the  target  x-axis  with  respect  to  the  horizontal  plane 
and  is  positive  when  it  is  upward  (i.e.,  a  nose-down  target  has  a  pitch  angle  of  -90°).  The 
conventions  for  roll  are  harder  to  describe  in  words.  In  any  case,  roll  is  meaningless  for  a  target 
having  an  axis  of  symmetry. 

Target  principal  polarizability  rate  transients:  These  are  three  transient  decay  curves 
representing  the  principal  polarizability  rates  (dP(t)/dt)  at  each  of  the  time  gates  observed  with 
the  data.  At  APG,  we  acquired  static  data  at  a  base  frequency  of  30  Hz  resulting  in  transient 
decay  lengths  of  8,333  ps.  These  transients  were  decimated  into  42  logarithmically  spaced  time 
gates  with  centers  ranging  from  106  to  7,912  ps.  The  transients  are  ordered  so  that  the  largest 
corresponds  with  the  x-axis  of  the  principal  system  and  the  smallest  corresponds  with  the  z-axis. 
The  units  of  polarizability  rate  is  m  /s  =  cm  /ps. 

Integrated  target  polarizability  (Pox,  Poy,  and  POz):  These  three  parameters  represent  the 
numerical  integration,  respectively,  of  the  three  polarizability  rate  transients’  curves  (5  above). 
Because  the  integration  is  with  respect  to  time,  the  resulting  units  of  integrated  polarizability  will 
be  either  m3  or  cm3  depending  on  the  units  of  time  used.  Time  units  of  ps  were  used  in  the 
analyses  and  therefore,  the  units  of  target  polarizability  are  cm  .  These  parameters  provide  a 
sense  of  target  size.  From  these  three  parameters,  we  can  further  define  a  couple  of  other  useful 
parameters,  which  are  R  =  2Pox/(Poy  +  Poz)  and  E  =  (Poy  -  Poz)/Pox.  The  R  parameter  indicates  the 
elongation  or  aspect  of  the  resulting  target.  When  R  >  1 ,  we  have  an  elongated  (rod-like)  target. 
The  E  parameter  is  a  measure  of  symmetry.  When  the  two  minor  polarizability  curves  (dPoy/dt 
and  dP0z/dt)  are  identical,  E  =  0. 

b.  Why  have  these  characteristics  been  chosen  and  not  others  (e.g.,  empirical  evidence  of 
their  ability  to  help  discriminate,  inclusion  in  a  theoretical  tradition,  etc.)? 

These  parameters  were  chosen  because  the  shape  and  amplitude  of  the  polarizability  rate 
transients  five  contain  all  available  characteristics  of  the  targets.  The  integrated  parameters  six 
in  fact  are  very  robust  metaparameters  that  can  be  derived  directly  from  the  curves.  Based  on 
our  experience  with  the  analysis  of  targets  from  YPG  along  with  the  knowledge  that  at  APG  only 
a  maximum  of  six  targets  to  identify,  it  is  believed  that  discrimination  could  best  be 
accomplished  through  matching  curve  characteristics  of  unknown  targets  with  the  characteristics 
of  a  library  of  “type”  curves  generated  through  analysis  of  target  polarizabilities  measured  in  the 
calibration  lanes. 

c.  How  are  these  characteristics  estimated  (e.g.,  least-mean-squares  fit  to  a  dipole  model, 
etc.),  include  the  equations  that  are  used  for  parameter  estimation? 
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These  characteristics  are  derived  from  a  series  of  three-step  least-squares  estimation.  The 
figure  shows  only  a  single  transmitter  loop  and  a  single  receiver  loop.  The  data  set  acquired  by 
the  Metal  Mapper  in  fact  involves  three  different  transmitter  loops  and  21  different  receiver 
loops.  The  three  estimation  steps  are: 

Estimate  position  and  composite  polarizability  tensor  using  a  composite  gate  on  each 
transient,  thereby  converting  the  63  transients  (21  transients  for  each  of  three  transmitters)  to 
63  scalar  gate  values).  This  is  a  nonlinear  problem  in  nine  unknowns. 

Estimate  target  attitude  by  doing  an  eigen  analysis  of  the  polarization  tensor  resulting 
from  the  application  of  step  1 .  This  is  a  linear  problem  in  three  unknowns. 

Estimate  the  principal  polarizability  transient  values  at  each  time  gate.  This  is  a  linear 
least  squares  problem. 

d.  What  tunable  parameters  (if  any)  are  used  in  the  characterization  process  (e.g., 
thresholds  on  background  noise,  etc.)? 

None.  The  inversion  requires  that  we  provide  a  background  data  set  that  is  to  be 
subtracted  from  the  observed  data  set  prior  to  performing  the  inversion.  This  background  data 
set  is  optional  and,  in  cases,  where  the  SNR  of  the  observed  data  is  large  (e.g.,  40  dB)  there  is  no 
practical  difference  between  parameters  extracted  after  background  has  been  removed  and 
parameters  extracted  without  first  removing  background.  However,  at  lower  SNR  levels 
(i.e.,  <30  dB),  the  removal  of  background  is  important.  For  that  reason,  a  background  is  always 
removed.  Moreover,  a  background  data  file  is  acquired  at  a  calibration  site  2  to  3  times  per  day 
and  the  most  current  background  data  file  (i.e.,  the  background  file  whose  acquisition  time  is 
closest  to  the  data  file  under  investigation)  is  used  for  the  inversion.  The  inversion  process  also 
requires  a  data  file  that  provides  an  estimate  of  the  RMS  noise  levels  at  every  time  gate  and  for 
every  receiver  channel.  These  noise  levels  are  used  to  estimate  errors  in  the  parameter  estimates. 
The  RMS  noise  file  is  not  critical  for  good  inversion  results.  However,  in  the  field  procedure 
data  is  acquired  for  a  noise  estimate  every  time  static  background  is  measured  (2  to  3  times  per 
day). 


Classification. 

a.  What  algorithm  is  used  for  discrimination  (e.g.,  multilayer  perception,  support  vector 
machine,  etc.)? 

A  library  curve  matching  method  to  affect  discrimination  was  used.  For  each  of  the 
three  blind  areas,  the  library  consisted  of  the  relevant  target  parameters  for  the  three  (or  six  for 
the  blind  grid)  seeded  target  types. 
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b.  Why  is  this  algorithm  used  and  not  others? 

During  the  analysis  of  the  target  data  acquired  over  the  calibration  lanes  at  APG,  two 
approaches  were  evaluated.  In  the  first  approach,  multivariate  statistics  were  used  to  identify  a 
set  of  target  features  based  on  two  sets  of  integrated  polarizability  parameters:  the  Po  parameters 
described  in  six  of  the  previous  section,  and  a  second  set  (Pix,  Piy,  Piz)  that  represent  numerically 
derived  values  for  the  first  moments  in  time  of  the  respective  polarizability  rate  transients  (five  in 
previous  section).  The  first  moments  (Pix,  Piy,  and  Plz)  weight  the  late  times.  It  was  found  that 
the  algorithm  worked  well  on  the  training  data  but  was  unsatisfactory  when  applied  to  blind  data. 

The  second  method  experimented  with  was  curve  matching.  This  method  works  very 
well  on  training  data  and,  based  on  preliminary  results  provided  by  ATC,  also  worked  well  on 
data  from  the  blind  areas  that  were  surveyed.  Curve  matching  was  chosen  because  it  appeared  to 
work  better  on  blind  data  as  judged  when  compared  to  the  discrimination  result  provided  by 
numerical  curve  matching  and  a  visual  inspection  polarization  curves  with  the  candidate  “type” 
curves  (an  optical  correlation  performed  by  an  expert). 

c.  Which  parameters  are  considered  as  possible  inputs  to  the  algorithm? 

Figure  of  merit  (FOM):  This  parameter  is  derived  from  the  SNR  amplitude 
characteristic  of  the  data  set.  It  recognizes  the  fact  when  the  SNR  of  the  data  set  is  greater  than 
10  (20  dB),  the  resulting  parameters  from  the  inversion  estimates  are  good.  At  lower  SNRs,  the 
results  are  less  reliable.  The  parameter  (SNR  =10)  has  been  established  over  a  period  of  about 
8  years,  during  which  four  different  algorithms  for  the  estimation  of  target  parameters  have  been 
used.  While  differing  somewhat  in  computational  flow,  these  four  algorithms  all  suggest  that 
reliable  results  require  a  SNR  >10.  The  formula  for  FOM  is: 


FOM[SNR;  SNRlh  ]  =  Max 


0,1  +  Min 


0,log10 


SNR 


FOM  takes  the  value  1  for  all  SNR>  =10,  and  0  for  all  SNR<  1.  It  was  used  as  a 
multiplicative  penalty  function  to  derate  the  discrimination  score  based  on  the  fact  that  the  data 
set  has  a  low  SNR. 


Fit  (fitd):  The  Fit  statistic  is  an  indicator  of  the  quality  of  the  underlying  parameters.  A 
desired  discrimination  metric  contain  a  penalty  for  poor  fits.  Therefore,  we  defined  the  function: 


fitd[fit;a,b] 


1 

1 00  -  fit  -  b 


\+e  Cl 
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Fitd  ~  1  for  good  fits  (fit  ~  100)  and  Fitd-  0  for  poor  fits  (fit  ~  0).  The  rate  of  transition 
is  controlled  by  the  parameter  a.  The  point  at  which  Fitd =  0.5  is  controlled  by  the  parameter  b. 
For  discrimination  at  APG,  a  =  7,  and  b  =  28  was  used  for  these  parameters.  In  this  function, 
significant  penalties  begin  to  be  applied  for  fits  less  than  90  percent  and  at  fit  =  72  percent,  the 
penalty  is  0.5. 

Curve  Match  (log  Px):  This  operation  actually  supplies  two  parameters:  K  =  gain 
factor;  a  curve  shape  factor  shapex.  Normally,  polarizability  curves  are  displayed  on  a  log-log 
plot.  The  adopted  approach  that  the  curve  matching  should  follow  a  methodology  that  is  similar 
to  how  visually  compare  two  sets  of  principal  polarization  curves.  Therefore,  the  polarizability 
curves  were  matched  after  taking  their  logarithms.  Recognizing  that  it  is  possible  to  have  two 
polarization  curves  with  virtually  identical  shapes  but  different  amplitudes,  a  relative  gain  factor 
between  the  unknown  transient  curve  and  the  “type”  curve  needs  to  be  determined.  This  is  the 
parameter  K.  The  shape  factor  (shapex)  represents  in  an  RMS  sense  the  difference  between  the 
logarithms  of  the  unknown  transient  and  its  corresponding  “type”  transient  after  first  correcting 
the  unknown  transient  for  a  gain  factor  K.  An  exact  match  between  two  transient  curves  is 
indicated  when  K  =  1  and  shapex=  100. 

Curve  Match  (log  R):  This  operation  compares  the  ratio  of  the  major  polarizability 
transient  (dPx/dt)  to  the  arithmetic  (or  geometric)  average  of  the  two  minor  polarizabilities 
(dPy/dt  and  dPz/dt).  Since  the  R  parameter  represents  a  ratio,  constant  gain  factors  are  cancelled 
out.  Therefore,  in  this  curve  match,  the  RMS  error  is  computed  between  the  logarithm  of  the 
unknown  ratio  curve  and  one  of  the  “type”  ratio  curves.  A  return  of  this  parameter  (shaped  is  a 
number  between  0  and  100,  with  100  being  a  perfect  match. 

Size  and  Shape  Metric  (M):  The  three  parameters  defined  above  were  used  to  define 
the  following  metric  that  reflects  a  match  of  both  size  (i.e.,  sameness  in  amplitude)  and  shape 
(similarity  in  logarithmic  shape). 


M  [K,  shape x ,  shape  R ;  a  x 


,w  ,wD  =  w  e 

1  x  9  R  X 


log  K 


shape  x  +  wRshapeR 


The  third  equation  above  is  simply  a  weighted  sum  of  the  two  shape  factors  generated  in 
numbers  above.  The  weighting  parameters  (wx  and  Wr)  reflect  provided  the  ability  to  weight  the 
two  shape  parameters  differently.  Lacking  any  indication  of  whether  one  was  in  some  sense 
better  than  the  other,  the  value  wx  =  wx=  1/2  was  used.  Because  the  match  between  the  major 
polarity  curve,  a  corresponding  library  “type”  curve,  is  a  function  not  only  of  logarithmic  shape 
but  also  gain,  a  multiplicative  penalty  factor  was  introduced  into  the  first  term  in  equation  three. 
That  term  will  be  recognized  by  most  readers  as  a  Gaussian-shaped  curve  having  a  value  of 
1  when  logK=l  and  dropping  to  a  value  of  0.61  when  logK=cx.  Empirically,  it  was 
determined  that  a  value  of  cx=  0.25  worked  well  when  metric  was  trained  on  data  from  the  APG 
calibration  grid. 
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Eccentricity  Factor  (1-E):  Eccentricity  in  six  of  the  previous  sections  was  defined. 
This  parameter  is  important  since  it  reflects  the  likeness  of  the  two  minor  principal  polarizability 
transients.  When  the  eccentricity  is  very  low,  an  axis  of  symmetry  to  the  target  that  is  directed 
along  the  target  X  axis  is  ascribed.  Likewise,  high  eccentricity  is  generally  attributed  as  the  most 
distinguishing  feature  of  a  clutter  target.  This  is  not  always  the  case,  however,  since  the  ability 
of  the  physics-based  modeling  to  adequately  distinguish  symmetry  degrades  with  SNR  and  also 
with  certain  attitude  situations  (e.g.,  45°  pitch  angles).  So  the  parameter  is  applied  (1-E)  as  a 
penalty  function  in  the  metric  to  downgrade  otherwise  good  scores. 

The  METRIC  (Md):  Equations  1  through  3  were  combined  along  with  the  eccentricity 
factor  described  above  to  form  a  metric  that  provides  values  ranging  from  0  to  100.  A  metric 
value  was  generated  for  each  of  the  3/6  target  types  in  our  library  depending  on  the  area  that  was 
analyzed.  The  target  class  was  identified  according  to  the  “type”  curves  that  generated  the  best 
(maximum)  metric  value. 

M d  =  FOM\SNR;  SNRt/l  ]fitd  [ fit ;  a,b]M[K, shape x ,  shape R ;  cjx  ,  wx ,  wR  ](l  -  E) 

d.  What  are  the  outputs  of  the  algorithm  (probabilities,  confidence  levels)? 

The  output  (Md)  of  the  discrimination  metric  is  scaled  to  have  positive  values  ranging 
from  0  <  Md<  100.  These  values  certainly  reflect  relative  confidence,  but  in  no  way  can  they  be 
construed  as  probabilities  (expressed  as  a  percentage).  Curve  matching  often  does  not  produce 
sharp  contrasts  between  targets  of  the  same  general  size  but  belong  to  different  classes 
(e.g.,  25-mm  (DF)  versus  37-mm  (DF);  105-mm  (IF)  versus  105-mm  (DF)).  Similarly,  on  the 
basis  of  shape  and  size  alone,  there  were  clutter  targets  that  produced  high  discrimination 
metrics.  Only  by  introducing  the  eccentricity  factor  (1-E)  were  we  able  to  move  these  targets 
into  the  clutter  class. 

e.  How  is  the  threshold  set  to  decide  where  the  munitions/nonmunitions  line  lies  in  the 
discrimination  process? 

A  threshold  is  set  by  visually  examining  the  resulting  metric  values  (together  with  the 
values  for  their  underlying  components  -FOM,  fitd,  M,  and  E)  and  the  corresponding 
polarizability  curves.  At  some  low  value  of  the  metric  Md  it  became  obvious  that  the  target  was 
no  longer  ordnance.  This  threshold  was  different  for  each  of  the  three  areas  analyzed,  as  to  why 
is  uncertain.  The  thresholds  for  the  IF  and  DF  areas  were  relatively  close  together  (21.0  and 
14.6,  respectively),  so  these  two  target  lists  were  combined  without  in  any  way  adjusting  the 
thresholds.  However,  the  discrimination  threshold  in  the  blind  grid  was  much  higher.  No 
explanation  for  that  fact  other  than  to  note  that  the  targets  in  the  blind  grid  were  separated  by  a 
minimum  distance  of  2  meters.  As  a  consequence,  there  was  much  less  interference.  The 
inference  is  that  the  fit  quality  as  indicated  by  such  indicators  as  the  fitd  and  (1-E)  produce 
consistently  better  discrimination  values  in  the  blind  grid. 
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Training. 


a.  Which  tunable  parameters  have  final  values  that  are  optimized  over  a  training  set  of 
data  and  which  have  values  that  are  set  according  to  geophysical  knowledge  (i.e.,  intuition, 
experience,  common  sense)? 

Of  the  six  tunable  parameters  identified  in  equation  4,  only  the  parameter  ox  was 
determined  using  the  training  data.  Values  for  the  other  five  (SNRth,  a,  b,  wx,  and  Wr)  were  set 
with  geophysical  knowledge. 

(1)  For  those  tunable  parameters  with  final  values  set  according  to  geophysical 
knowledge: 

(a)  What  is  the  reasoning  behind  choosing  these  particular  values? 

SNR*  was  chosen  based  on  both  vendor  and  published  results  suggesting  that  at  SNR 
levels  below  10,  the  quality  of  the  inversion  diminishes  significantly.  The  values  chosen  were  of 
the  parameters  a  and  b  in  fitd.  The  parameters  were  such  that  there  is  virtually  no  penalty  for 
solutions  for  a  fit  statistic  above  90  percent.  Thereafter,  the  penalty  increases  rapidly.  Inversion 
parameter  sets  below  a  level  of  between  60  to  70  percent  can  provide  some  rough  indication  of 
depth  and,  correspondingly  size.  This  information  is  sometimes  sufficient  to  exclude  a  target 
(based  on  size  and  depth)  not  wanting  to  totally  exclude  inversion  results  with  low  fit  scores. 
With  regard  to  the  shape  parameters  (shapex,  and  shapeR),  the  maximum  entropy  approach  was 
adopted  and  it  was  decided  that  the  shape  of  the  primary  polarizability  (along  with  its  gain)  and 
the  shape  of  the  ratio  curve  were  equally  important.  This  gave  each  of  the  parameters 
(wx,  and  wr)  equal  weights. 

(b)  Why  were  the  final  values  not  optimized  over  a  training  set  of  data? 

In  the  case  of  the  first  three  parameters  (SNRth,  a,  and  b),  the  training  data  are  simply 
not  relevant  to  the  setting  of  these  data.  The  shape  parameter  weights  were  varied  (shapex  and 
shapeR)  about  the  chosen  uniform  values  using  trial  and  error.  The  results  were  relatively 
insensitive  to  small  changes.  Intuitively,  it  is  believed  that  the  ratio  curve  provides  important 
information  about  the  target.  The  25 -mm  (DF)  is  an  example  where  the  ratio  provides  a  very 
distinguishing  characteristic.  In  other  cases,  the  ratio  is  less  important.  It  was  decided  to  weigh 
the  two  characteristics  equally. 

(2)  For  those  tunable  parameters  with  final  values  optimized  over  the  training  set  data: 

(a)  What  training  data  is  used  (e.g.,  all  data,  a  randomly  chosen  portion  of  data,  etc.)? 

All  data  was  used  from  the  calibration  lanes  including  grid  sites  that  contained 
munitions  that  were  not  one  of  the  six  types  of  interest. 

(b)  What  error  metric  is  minimized  during  training  (e.g.,  mean  squared  error,  etc.)? 
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A  simple  binary  decision  (correct  ID  =  1 ;  incorrect  ID  =  0)  was  used,  simply 
minimizing  the  number  of  misidentified  targets 

(c)  What  learning  rule  is  used  during  training  (e.g.,  gradient  descent,  etc.)? 

A  trial  and  error  to  change  the  parameter  (gx). 

(d)  What  criterion  is  used  to  stop  training  (e.g.,  number  of  iterations  exceeds  threshold, 
good  generalization  over  validation  set  of  data,  etc.)? 

Stopping  training  when  the  number  of  incorrect  decisions  was  minimized. 

(e)  Are  all  tunable  parameters  optimized  at  once  or  in  sequence  (in  sequence  =  parameters 
1  is  held  constant  at  some  common  sense  values  while  parameter  2  is  optimized,  and  then 
parameter  2  is  held  constant  at  its  optimized  value  while  parameter  1  is  optimized)? 

This  question  is  not  really  applicable  to  what  was  done.  However,  to  the  extent  that 
played  a  bit  with  optimizing  the  shape  weights,  this  optimization  was  done  in  sequence  holding 
ox  constant  while  varying  the  weights.  Note  that  since  the  weights  have  a  constraint 
(i.e.,  wx  2  +  wr  2  =  1),  in  essence  is  varying  only  a  single  parameter. 

b.  What  are  the  final  values  of  all  tunable  parameters  for  the  characterization  process? 


SNR*  =  10 
a  =  7;  b  =  28 


ox=  0.25;  wx=  Wr=  1/2 


2.1.4  Data  Submission  Format 


Data  were  submitted  for  scoring  in  accordance  with  data  submission  protocols  outlined  on 
the  USAEC  Web  site  www.uxotestsites.org.  These  submitted  data  are  not  included  in  this  report 
in  order  to  protect  GT  information. 

2.1.5  Demonstrator  Quality  Assurance  (QA)  and  Quality  Control  (QC)  (provided  by 

demonstrator) 

Quality  Control  (QC).  The  AOL2  DAQ  system  integrates  data  acquired  from  three 
(optionally  4)  sensors  into  a  sample  data  point.  These  systems  are:  position;  attitude;  EM,  and 
(optionally)  MAG.  The  data  from  each  of  the  systems  are  integrated  into  a  single  data  structure 
(i.e.,  an  EM3DDataPoint).  Performed  system  checks  by  returning  to  a  calibration  point  to 
acquire  data  will  occur.  Typically,  the  system  check  consists  of  a  short  profile  (approximately 
10  m)  that  is  surveyed  repeatedly  two  or  more  times  a  day.  The  profile  will  be  set  up  in  an  area 
of  typical  background  response  (i.e.,  no  targets).  The  calibration  survey  will  consist  of  a 
dynamic  survey  run  over  a  calibration  target  (typically  a  shot  put)  centered  along  the  profile.  At 
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the  start  of  the  calibration  survey  a  static  point  using  both  dynamic  and  static  acquisition 
parameters  at  the  beginning  of  the  calibration  line  is  acquired,  the  target  is  surveyed  dynamically 
in  one  direction,  and  then  the  survey  is  repeated  in  the  opposite  direction.  Finally,  the  antenna 
array  is  halted  directly  over  the  target  and  acquires  a  static  data  point.  The  static  points 
(static/dynamic  parameters)  provide  base-level  background  measurements.  These  measurements 
are  useful  in  determining  whether  the  background  changes  significantly  over  the  area  of  the 
survey.  The  calibration  survey  lines,  repeated  in  opposite  directions,  provide  a  check  of  survey 
timing  latency  between  the  acquisition  of  the  GPS  position  and  the  acquisition  of  the  EM  data. 
Position  latencies  typical  of  systems  where  survey  positions  and  data  are  merged  from 
independent  data  files  based  on  a  time  stamp  has  not  been  experienced  because  of  the  way  the 
GPS  position  is  integrated  directly  with  the  data.  However,  this  experiment  provides 
proof-positive  that  there  is  no  significant  timing  latency  in  the  acquisition  system.  The 
amplitude  of  the  dynamic  survey  peaks  as  they  cross  over  the  calibration  target  and  also  provides 
a  crude  measure  of  the  EM  drift.  A  better  measure  of  the  drift  is  provided  by  the  static 
measurements  of  the  background  and  the  target  response.  As  part  of  the  static  background 
measurement,  a  precise  method  for  putting  the  cart  into  a  known  and  repeatable  attitude  will  be 
established  so  that  the  reliability  of  the  orientation  system  may  be  checked.  It  is  notable  that  the 
DAQ  system  constantly  monitors  the  quality  of  the  GPS  positions  and  provides  a  visual  warning 
to  the  operator  when  the  GPS  quality  for  any  reason  degrades  below  that  of  real-time  kinematic 
(RTK).  Furthermore,  the  acquisition  software  includes  the  ability  to  graphically  display  data 
from  any  point  in  any  data  file.  This  plotting  capability  allows  data  to  be  checked  at  anytime 
while  in  the  field. 

Overview  of  Quality  Assurance  (QA). 

a.  Three  main  objectives  for  the  demonstration  at  ATC: 

(1)  Demonstrate  that  the  Metal  Mapper  system  represents  a  significant  advance  in  the 
state-of-the-art.  This  will  be  done  by  demonstrating  that  both  hardware  and  software  perfonns  as 
well  as  other  next  generation  EMI  systems  commonly  accepted  to  represent  an  advance  in  the  art 
(ALLTEM,  BUD,  and  the  NRL  TEM  Array  system). 

(2)  .  Demonstrate  that  the  Metal  Mapper  system  is  ready  to  conduct  product  UXO  surveys. 

.  (3)  Using  dynamic  data  collected  from  the  APG  survey,  determine  the  configuration  for  a 
base-level  Metal  Mapper  system.  The  resulting  configuration  will  be  used  primarily  for  mapping 
and  will  compete  with  presently  used  EMI  technology  such  as  the  EM61-MKII  daily  at  the  start 
of  the  field  day,  at  mid-day,  and  at  quitting  time. 

b.  Dynamic  Survey.  Dynamic  surveys  over  both  the  calibration  grid  and  the  blind  test 
grid  will  be  conducted.  These  surveys  will  be  conducted  using  excitation  with  a  single 
transmitter  loop  at  1 -meter  lane  intervals.  All  or  part  of  these  surveys  may  be  repeated  using 
different  acquisition  parameters.  The  maps  that  are  compiled  from  these  data  will  be  used  for 
target  detection. 
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c.  Calibration  Checks.  Proper  functioning  of  both  navigation  and  EM  data  acquisition 
will  be  assured  by  conducting  periodic  calibration  surveys  as  described  earlier.  These  surveys 
provide  a  check  of  the  three  critical  AOL2  subsystems,  navigation,  attitude,  and  EM  data 
acquisition,  as  well  as  serving  as  a  means  to  sample  long  term  drift  of  the  instrument  response. 

d.  Static  Surveys.  Using  the  target  list  generated  from  the  dynamic  surveys  above,  each 
target  will  be  reacquired,  and  a  static  data  set  will  be  taken  that  consists  of  the  EMI  response 
from  all  three  transmitter  polarizations.  For  static  measurements,  DAQ  parameters  will  be 
changed  to  allow  the  acquisition  of  a  longer  time  transient  (e.g.,  T  =  0.3s,  N  =  9  to  provide  us 
with  a  8.3  ms  transient  decay).  The  acquisition  stack-count  is  generally  set  so  that  a  data  point  is 
acquired  in  less  than  30  sec.  Previous  experience  with  this  survey  mode  has  demonstrated  that  it 
is  capable  of  acquiring  up  to  200  targets  per  day.  As  with  the  dynamic  surveys,  a  repeat  of  all  or 
part  of  the  two  grids  using  different  acquisition  parameters  may  occur. 

e.  Calibration  Checks.  All  static  surveys  will  include  periodic  measurements  at  a 
background  site  and  over  a  calibration  target.  Furthermore,  the  intent  is  to  acquire  static 
background  points  at  random  locations  within  the  calibration  and  blind  grids  that  are  judged  to 
be  background  with  the  objective  of  determining  whether  background  varies  with  position.  The 
frequency  of  the  calibration  checks  will  depend  on  the  drift  rates  that  are  observed  during 
surveys  over  the  calibration  grid.  At  a  minimum,  however,  these  calibration  checks  will  be  run 
two  times  daily  at  the  start  of  the  field  day  and  at  quitting  time. 

2.1.6  Additional  Records 

The  following  record(s)  by  this  vendor  can  be  accessed  via  the  Internet  as  Microsoft  Word 
documents  at  www .uxotestsites .  or g . 
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2.2  APG  SITE  INFORMATION 


2.2.1  Location 


The  APG  Standardized  Test  Site  is  located  within  a  secured  range  area  of  the  Aberdeen 
Area.  The  Aberdeen  Area  of  APG  is  located  approximately  30  miles  northeast  of  Baltimore  at 
the  northern  end  of  the  Chesapeake  Bay.  The  Standardized  Test  Site  encompasses  17  acres  of 
upland  and  lowland  flats,  woods,  and  wetlands. 

2.2.2  Soil  Type 

According  to  the  soils  survey  conducted  for  the  entire  area  of  APG  in  1998,  the  test  site 
consists  primarily  of  Elkton  Series  type  soil  (ref  2).  The  Elkton  Series  consist  of  very  deep, 
slowly  permeable,  poorly  drained  soils.  These  soils  formed  in  silty  aeolin  sediments  and  the 
underlying  loamy  alluvial  and  marine  sediments.  They  are  on  upland  and  lowland  flats  and  in 
depressions  of  the  Mid-Atlantic  Coastal  Plain.  Slopes  range  from  0  to  2  percent. 

ERDC  conducted  a  site-specific  analysis  in  May  2002  (ref  3).  The  results  basically 
matched  the  soil  survey  mentioned  above.  Seventy  percent  of  the  samples  taken  were  classified 
as  silty  loam.  The  majority  (77  percent)  of  the  soil  samples  had  a  measured  water  content 
between  15  and  30  percent  with  the  water  content  decreasing  slightly  with  depth. 

For  more  details  concerning  the  soil  properties  at  the  APG  test  site,  go  to 
www.uxotestsites.org  on  the  Web  to  view  the  entire  soils  description  report. 

2.2.3  Test  Areas 


A  description  of  the  test  site  areas  at  APG  is  presented  in  Table  1.  A  test  site  layout  is 
shown  in  Figure  2. 
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TABLE  1.  TEST  SITE  AREAS 


Area 

Description 

Calibration  lanes 

Contains  14  standard  munitions  items  buried  in  six  positions,  with  representation 
of  clutter,  at  various  angles  and  depths  to  allow  demonstrators  to  calibrate  their 
equipment. 

Blind  grid 

Contains  400  grid  cells  in  a  0.5-acre  site.  The  center  of  each  grid  cell  contains 
either  munitions,  clutter,  or  nothing. 

Open  field 

A  10-acre  site  composed  of  generally  open  and  flat  terrain  with  minimal  clutter 
and  minor  navigational  obstacles.  Vegetation  height  varies  from  15  to  25  cm. 
This  area  is  subdivided  into  four  subareas  (legacy,  direct  fire,  indirect  fire,  and 
challenge). 

•  Open  field  (legacy) 

The  legacy  subarea  contains  the  same  wide  variety  of  randomly-placed  munitions 
that  were  present  in  the  open  field  prior  to  the  January  2008  general 
reconfiguration  of  the  site. 

•  Open  field  (direct  fire) 

The  direct  fire  subarea  contains  only  three  munition  types  that  could  be  typically 
found  at  an  impact  area  of  a  direct  fire  weapons  range.  Munitions  and  clutter  are 
placed  in  a  pattern  typical  for  these  munitions. 

•  Open  field  (indirect  fire) 

The  indirect  fire  subarea  contains  only  three  munition  types  that  could  be  typically 
found  at  an  impact  area  of  an  indirect  fire  weapons  range.  Munitions  and  clutter 
are  placed  in  a  pattern  typical  for  these  munitions. 

•  Open  field  (challenge) 

The  challenge  subarea  is  easily  reconfigurable  used  to  meet  the  specific  needs  and 
requirements  of  the  demonstrator  or  the  program  sponsor.  Any  results  from  this 
area  will  not  be  reported  in  the  standardized  scoring  record. 

Woods 

1.34-acre  area  consisting  of  cleared  woods  (tree  removal  with  only  stumps 
remaining),  partially  cleared  woods  (including  all  underbrush  and  fallen  trees), 
and  virgin  woods  (i.e.,  woods  in  natural  state  with  all  trees,  underbrush,  and 
fallen  trees  left  in  place). 

Moguls 

1.30-acre  area  consisting  of  two  areas  (the  rectangular  or  driving  portion  of  the 
course  and  the  triangular  section  with  more  difficult,  nondrivable  terrain).  A 
series  of  craters  (as  deep  as  0.91  m)  and  mounds  (as  high  as  0.91  m)  encompass 
this  section. 

2.2.4  STANDARD  AND  NONSTANDARD  INERT  MUNITIONS  TARGETS 

The  standard  and  nonstandard  munitions  items  emplaced  in  the  test  areas  are  presented  in 
Table  2.  Standardized  targets  are  members  of  a  set  of  specific  munitions  items  that  have 
identical  properties  to  all  other  items  in  the  set  (caliber,  configuration,  size,  weight,  aspect  ratio, 
material,  filler,  magnetic  remanence,  and  nomenclature).  Nonstandard  targets  are  inert 
munitions  items  having  properties  that  differ  from  those  in  the  set  of  standardized  items. 
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TABLE  2.  INERT  MUNITIONS  TARGETS 


Munition 

Calibration 

Open  Field 

Open  field 

Open  Field 

Item 

Type 

Lanes 

Blind  Grid 

Direct  Fire 

Indirect  Fire 

Legacy 

Moguls 

Woods 

20-mm  Projectile  M55 

S 

X 

X 

X 

X 

25-mm  Projectile  M794 

S 

X 

X 

X 

37-mm  Projectile  M47 

s 

X 

X 

X 

40-mm  Projectile  MKII  Bodies 

s 

X 

X 

X 

X 

BDU-28  Submunition 

s 

X 

X 

X 

X 

BLU-26  Submunition 

s 

X 

X 

X 

X 

M42  Submunition 

s 

X 

X 

X 

X 

57-mm  Projectile  APC  M86 

s 

X 

X 

X 

X 

60-mm  Mortar  M49A3 

s 

X 

X 

X 

2.75-in.  Rocket  M230 

s 

X 

X 

X 

X 

81 -mm  Mortar  M3  74 

s 

X 

X 

X 

X 

X 

X 

105-mm  HEAT  Rounds  M456 

s 

X 

X 

X 

105-mm  HEAT  Round  M490 

s 

X 

X 

X 

105-mm  Projectile  M60 

s 

X 

X 

X 

X 

X 

X 

155-mm  Projectile  M483A1 

s 

X 

X 

X 

X 

20-mm  Projectile  M55 

NS 

X 

X 

X 

20-mm  Projectile  M97 

NS 

X 

X 

X 

40-mm  Projectile  M813 

NS 

X 

X 

X 

60-mm  Mortar  (JPG) 

NS 

X 

X 

X 

60-mm  Mortar  M49 

NS 

X 

X 

X 

2.75-in.  Rocket  M230 

NS 

X 

X 

X 

2.75-in.  Rocket  XM229 

NS 

X 

X 

X 

81 -mm  Mortar  (JPG) 

NS 

X 

X 

X 

81 -mm  Mortar  M3  74 

NS 

X 

X 

X 

105-mm  Projectile  M60 

NS 

X 

X 

X 

155-mm  Projectile  M483A 

NS 

X 

X 

X 

S 

NS 

JPG 

HEAT 


Standard  munition. 
Nonstandard  munition. 
Jefferson  Proving  Ground, 
high-explosive  antitank. 


Figure  2.  Test  site  layout. 
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(Page  26  Blank) 


SECTION  3.  FIELD  DATA 


3.1  DATE  OF  FIELD  ACTIVITIES  (15  to  20,  22  to  27,  30  September;  1,  2,  and  4  October  2008) 

3.2  AREAS  TESTED/NUMBER  OF  HOURS 

Areas  tested  and  total  number  of  hours  operated  at  each  site  are  presented  in  Table  3. 


TABLE  3.  AREAS  TESTED  AND 
NUMBER  OF  HOURS 


Area 

Number  of  Hours 

Calibration  lanes 

12.16 

Blind  grid 

17.75 

Open  field 

94.33 

Woods 

0.00 

Mogul 

0.00 

Note:  Table  3  represents  the  total  time  spent  in  each  area. 


3.3  TEST  CONDITIONS 
3.3.1  Weather  Conditions 

An  APG  weather  station  located  approximately  1  mile  west  of  the  test  site  was  used  to 
record  average  temperature  and  precipitation  on  a  half  hour  basis  for  each  day  of  operation.  The 
temperatures  presented  in  Table  4  represent  the  average  temperature  during  field  operations  from 
0700  to  1700  hours,  while  precipitation  data  represents  a  daily  total  amount  of  rainfall.  Hourly 
weather  logs  used  to  generate  this  summary  are  provided  in  Appendix  B. 


TABLE  4.  TEMPERATURE/PRECIPITATION  DATA  SUMMARY 


Date,  08 

Average  Temperature,  °F 

Total  Daily  Precipitation,  in. 

15  Sep 

80.6 

0.00 

16  Sep 

68.4 

0.00 

17  Sep 

69.3 

0.00 

18  Sep 

72.3 

0.00 

19  Sep 

66.6 

0.00 

20  Sep 

65.1 

0.00 

22  Sep 

71.8 

0.00 

23  Sep 

67.8 

0.00 

24  Sep 

67.2 

0.00 

25  Sep 

62.0 

0.13 

26  Sep 

66.4 

0.11 

27  Sep 

71.8 

0.55 

27 


TABLE  4.  (CONT’D) 


Date,  08 

Average  Temperature,  °F 

Total  Daily  Precipitation,  in. 

30  Sep 

67.4 

0.33 

01  Oct 

59.6 

0.32 

02  Oct 

64.4 

0.00 

04  Oct 

62.7 

0.00 

3.3.2  Field  Conditions 


G&G  Sciences,  Inc.  surveyed  from  mid-September  through  early  October.  The 
temperatures  were  seasonable.  There  were  areas  of  standing  water  in  various  places  throughout 
the  open  field.  This  did  not  hinder  the  survey. 

3.3.3  Soil  Moisture 


Three  soil  probes  were  placed  at  various  locations  within  the  site  to  capture  soil  moisture 
data:  blind  grid,  calibration,  open  field,  and  wooded  areas.  Measurements  were  collected  in 
percent  moisture  and  were  taken  twice  daily  (morning  and  afternoon)  from  five  different  soil 
depths  (1  to  6  in.,  6  to  12  in.,  12  to  24  in.,  24  to  36  in.,  and  36  to  48  in.)  from  each  probe.  Soil 
moisture  logs  are  included  in  Appendix  C. 

3.4  FIELD  ACTIVITIES 

3.4.1  Setup/Mobilization 

These  activities  included  initial  mobilization  and  daily  equipment  preparation  and 
breakdown.  A  three-person  crew  took  2  hours  and  45  minutes  to  perform  the  initial  setup  and 
mobilization.  There  were  10  hours  and  50  minutes  of  daily  equipment  preparation,  and  end  of 
the  day  equipment  breakdown  lasted  5  hours  and  15  minutes. 

3.4.2  Calibration 


G&G  Sciences,  Inc.  spent  a  total  of  12  hours  and  10  minutes  in  the  calibration  lanes,  of 
which  6  hours  and  30  minutes  was  spent  collecting  data.  Numberous  calibration  activities 
occurred  while  surveying  the  open  field  and  blind  grid.  Total  amount  of  time  was  6  hours  and 
45  minutes. 

3.4.3  Downtime  Occasions 


Occasions  of  downtime  are  grouped  into  five  categories:  equipment/data  checks  or 
equipment  maintenance,  equipment  failure  and  repair,  weather,  demonstration  site  issues,  or 
breaks/lunch.  All  downtime  is  included  for  the  purposes  of  calculating  labor  requirements 
(section  5)  except  for  downtime  due  to  demonstration  site  issues.  Demonstration  site  issues, 
while  noted  in  the  daily  log,  are  considered  nonchargeable  downtime  for  the  purposes  of 
calculating  labor  costs  and  are  not  discussed.  Breaks  and  lunches  are  discussed  in  this  section 
and  billed  to  the  total  site  survey  area. 
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3.4.3. 1  Equipment/data  checks,  maintenance.  Equipment  data  checks  and  maintenance 
activities  accounted  for  3  hours  and  25  minutes  of  site  usage  time.  These  activities  included 
changing  out  batteries  and  perfonning  routine  data  checks  to  ensure  the  data  were  being  properly 
recorded/collected.  G&G  Sciences,  Inc.  spent  an  additional  5  hours  and  15  minutes  for  breaks 
and  lunches. 

3.4.3.2  Equipment  failure  or  repair.  No  time  was  needed  to  resolve  equipment  failures  that 
occurred  while  surveying. 

3.4.3.3  Weather.  No  weather  delays  occurred  during  the  survey. 

3.4.4  Data  Collection 


TABLE  5.  TOTAL  TIME  G&G  SCIENCES,  INC. 
SPENT  PER  AREA 


AREA 

Time,  hr/min 

Blind  grid 

1 7  hours/45  minutes 

Open  field 

94  hours/20  minutes 

Legacy 

N/A 

Direct  fire 

46  hours/ 15  minutes 

Indirect  fire 

48  hours/5  minutes 

Challenge 

N/A 

Wooded 

N/A 

Moguls 

N/A 

Note:  Table  5  represents  the  total  time  spent  in  each  area  collecting  data. 


3.4.5  Demobilization 


The  G&G  Sciences,  Inc.  survey  crew  went  on  to  conduct  a  full  demonstration  of  the  site. 
Therefore,  demobilization  did  not  occur  until  4  October  2008.  On  that  day,  it  took  the  crew 
55  minutes  to  break  down  and  pack  up  their  equipment. 

3.5  PROCESSING  TIME 

G&G  Sciences,  Inc.  submitted  the  raw  data  from  the  demonstration  activities  on  the  last 
day  of  the  demonstration,  as  required.  The  scoring  submittal  data  were  also  provided  within  the 
required  30-day  time  frame. 
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3.6  DEMONSTRATOR’S  FIELD  PERSONNEL 


Thomas  S.  King 
Donald  D.  Snyder 
David  C.  George 

3.7  DEMONSTRATOR’S  FIELD  SURVEYING  METHOD 

G&G  Sciences,  Inc.  surveyed  the  direct  fire,  indirect  fire,  and  blind  grid  in  a  linear  fashion 
with  3/4-meter  line  spacing.  G&G  Sciences,  Inc.  then  went  to  each  point  in  the  areas  where  they 
believed  something  was  buried  and  did  further  investigation  of  these  individual  points. 

3.8  SUMMARY  OF  DAILY  LOGS 

Daily  logs  capture  all  field  activities  during  this  demonstration  and  are  located  in 
Appendix  D.  Activities  pertinent  to  this  specific  demonstration  are  indicated  in  highlighted  text. 
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SECTION  4.  TECHNICAL  PERFORMANCE  RESULTS 


4.1  ROC  CURVES  USING  ALL  MUNITIONS  CATEGORIES 

The  probability  of  detection  for  the  response  stage  (Pdres)  and  the  discrimination  stage 
(P(idlsc)  versus  their  respective  probability  of  clutter  detection  or  probability  of  false  positive 
within  each  area  are  shown  in  Figures  3  through  8.  The  probabilities  plotted  against 
their  respective  background  alarm  rate  within  each  area  are  shown  in  Figures  9  through  14. 
Both  figures  use  horizontal  lines  to  illustrate  the  performance  of  the  demonstrator  at  two 
demonstrator-specified  points:  at  the  system  noise  level  for  the  response  stage,  representing  the 
point  below  which  targets  are  not  considered  detectable,  and  at  the  demonstrator’s  recommended 
threshold  level  for  the  discrimination  stage,  defining  the  subset  of  targets  the  demonstrator  would 
recommend  digging  based  on  discrimination.  Note  that  all  points  have  been  rounded  to  protect 
theGT. 


0  0.2  0.4  0.6  0.8  1 


Prob  cf  False  Positive 


-  -  Response 

- Qscrimnation 

- -  Noise  Level 

Threshold 


Figure  3.  Metal  Mapper/dual  mode  blind  grid  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 


31 


-  -  -  Response 

- □scririnalion 

*  -  -  •  Noise  Level 
■Threshold 


Figure  4.  Metal  Mapper/dual  mode  open  field  (direct  fire)  probability  of  detection  for  response 
and  discrimination  stages  versus  their  respective  probability  of  false  positive. 


-  -  -  Response 
- Kscrirrriation 

-  -  -  Nose  Level 

■Threshold 


Figure  5.  Metal  Mapper/dual  mode  open  field  (indirect  fire)  probability  of  detection  for  response 
and  discrimination  stages  versus  their  respective  probability  of  false  positive. 


32 


Not  covered 


Figure  6.  Metal  Mapper/dual  mode  open  field  (legacy)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  7.  Metal  Mapper/dual  mode  wooded  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  8.  Metal  Mapper/dual  mode  mogul  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 


-  -  *  Response 
- Discrimnation 

-  -  -  Noise  Level 

■Threshold 


Background  A I  ar  m  Rate 


Figure  9.  Metal  Mapper/dual  mode  blind  grid  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  background  alarm. 
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-  -  -  fesponse 
- Dscrirrination 

—  -  •  N)ise  Level 
- Threshold 


Figure  10.  Metal  Mapper/dual  mode  open  field  (direct  fire)  probability  of  detection  for  response 
and  discrimination  stages  versus  their  respective  background  alarm  rate. 


-  •  *  • fesponse 
- Dscrimnation 

—  -  •  N>ise  Level 

■  Threshold 


Figure  1 1.  Metal  Mapper/dual  mode  open  field  (indirect  fire)  probability  of  detection  for 

response  and  discrimination  stages  versus  their  respective  background  alarm  rate. 
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Not  covered 


Figure  12.  Metal  Mapper/dual  mode  open  field  (legacy)  probability  of  detection  for  response 
and  discrimination  stages  versus  their  respective  background  alarm  rate. 

Not  covered 

Figure  13.  Metal  Mapper/dual  mode  wooded  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  background  alann  rate. 

Not  covered 

Figure  14.  Metal  Mapper/dual  mode  mogul  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  background  alann  rate. 


4.2  PERFORMANCE  SUMMARIES 

Results  for  each  of  the  testing  areas  are  presented  in  Tables  6  (for  labor  requirements,  see 
section  5).  The  response  stage  results  are  derived  from  the  list  of  anomalies  above  the 
demonstrator-provided  noise  level.  The  results  for  the  discrimination  stage  are  derived  from  the 
demonstrator’s  recommended  threshold  for  optimizing  munitions  related  cleanup  by  minimizing 
false  alarm  digs  and  maximizing  munitions  recovery.  The  lower  and  upper  90-percent 
confidence  limits  on  Pd,  Pcd,  and  Pfp  were  calculated  assuming  that  the  number  of  detections  and 
false  positives  are  binomially  distributed  random  variables. 
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TABLE  6a.  BLIND  GRID  TEST  AREA  RESULTS 


j  Response  Stage 

Discrimination  Stage 

Munitions11 

Scores 

1  Pdes:  by  type  [ 

P/'“:  by  type 

All  Types 

105-mm 

81/60-mm 

37/25-mm 

All  Types 

105-mm 

81/60-mm 

37/25-mm 

1.00 

1.00 

0.98 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

0.99 

0.98 

0.94 

0.98 

0.93 

0.83 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

|  By  Depthb  | 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

0.96 

1.00 

0.92 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

8D  to  12D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

Clutter 

Scores 

Pcd 

Pfe 

j  By  Mass  j 

By  Depthb 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

0.97 

0.95 

0.91 

0.90 

1.00 

1.00 

0.03 

0.01 

0.00 

0.00 

0.00 

0.10 

0  to  0.15  m 

0.96 

0.93 

1.00 

1.00 

0.01 

0.00 

0.00 

0.17 

0.15  to  03  m 

0.88 

0.60 

1.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.3  to  0.6  m 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Background  Alann  Rates 

Pbares:  0.21  |  Pbadlsc:  0.02 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower 
90-percent  confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6b.  OPEN  FIELD  DIRECT  FIRE  TEST  AREA  RESULTS 


I  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pdes:  by  type  ! 

PJ"'C:  by  type 

All  Types 

105-mm 

37-mm 

2  5- mm 

All  Types 

105-mm 

37-mm 

25-mm 

0.90 

0.87 

0.84 

0.88 

0.83 

0.77 

0.91 

0.87 

0.81 

0.95 

0.87 

0.84 

0.78 

0.71 

0.89 

0.84 

0.78 

0.94 

0.90 

0.85 

0.91 

0.85 

0.86 

0.81 

[  By  Density  | 

High 

0.86 

0.92 

0.82 

0.85 

0.83 

0.84 

0.82 

0.85 

Medium 

0.83 

0.76 

0.79 

0.94 

0.80 

0.76 

0.72 

0.91 

Low 

0.92 

0.83 

0.97 

0.94 

0.90 

0.76 

0.97 

0.94 

|  By  Depthb  J 

0  to  4D 

0.91 

1.00 

0.86 

1.00 

0.88 

0.95 

0.83 

1.00 

4D  to  8D 

0.91 

0.93 

0.88 

0.91 

0.90 

0.91 

0.88 

0.91 

8D  to  12D 

0.68 

0.39 

N/A 

0.88 

0.61 

0.28 

N/A 

0.85 

Clutter 

Scores 

Pc 

Pfe 

|  By  Mass  ! 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

0.69 

0.65 

0.61 

0.47 

0.80 

0.08 

0.02 

0.08 

0.14 

0.92 

0.06 

0.04 

0  to  0.15  m 

0.65 

0.49 

0.80 

0.90 

0.05 

0.03 

0.09 

0.05 

0.15  to  03  m 

0.62 

0.39 

0.75 

0.88 

0.07 

0.00 

0.00 

0.25 

0.3  to  0.6  m 

0.80 

0.00 

1.00 

1.00 

0.20 

0.00 

0.00 

0.29 

Background  Alarm  Rates 

BARres:  0.07  |  BARdisc:  0.02 

Groups 

Found 

0.85 

0.52 

Identified 

0.03 

0.00 

Coverage 

0.44 

0.26 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6c.  OPEN  FIELD  INDIRECT  FIRE  TEST  AREA  RESULTS 


[  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pj™:  by  type  | 

Pj‘x:  by  type 

All  Types 

105- mm 

81-mrn 

60-mm 

All  Types 

105- mm 

81-mm 

60-mm 

0.93 

0.91 

0.88 

0.96 

0.93 

0.88 

0.92 

0.88 

0.82 

0.95 

0.91 

0.86 

0.90 

0.87 

0.84 

0.96 

0.92 

0.87 

0.83 

0.77 

0.70 

0.95 

0.91 

0.86 

|  By  Density  [ 

High 

0.86 

0.85 

0.96 

0.80 

0.85 

0.85 

0.91 

0.80 

Medium 

0.89 

0.93 

0.82 

0.90 

0.84 

0.90 

0.71 

0.90 

Low 

0.96 

1.00 

0.88 

1.00 

0.91 

1.00 

0.73 

1.00 

|  By  Depthb  [ 

0  to  4D 

0.96 

0.98 

1.00 

0.92 

0.94 

0.98 

0.91 

0.92  ; 

4D  to  8D 

0.91 

0.87 

0.92 

0.93 

0.82 

0.84 

0.77 

0.93 

8D  to  12D 

0.60 

0.75 

0.22 

0.83 

0.60 

0.75 

0.22 

0.83 

Clutter 

Scores 

Pc 

Pfe 

]  By  Mass  I 

By  Depthb 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

0.65 

0.62 

0.59 

0.47 

0.75 

0.05 

0.03 

0.02 

0.11 

0.87 

0.04 

0.02 

0  to  0.15  m 

0.61 

0.47 

0.76 

0.88 

0.03 

0.03 

0.02 

0.13 

0.15  to  03  m 

0.65 

0.38 

0.75 

0.82 

0.02 

0.00 

0.00 

0.06 

0.3  to  0.6  m 

0.83 

1.00 

0.67 

1.00 

0.17 

1.00 

0.00 

0.20  ! 

[  Background  Alarm  Rates  j 

[  Groups  | 

Found 

0.95 

0.80 

Identified 

0.00 

0.00 

Coverage 

0.47 

0.40 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6d.  OPEN  FIELD  LEGACY  TEST  AREA  RESULTS  (not  covered) 


|  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pj™:  by  type  | 

PJ"'C:  by  type 

All  Types 

Small 

Medium 

Large 

All  Types 

Small 

Medium 

Large 

- 

- 

- 

- 

- 

- 

- 

- 

|  By  Dept hh  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

- 

— 

— 

— 

— 

— 

8D  to  12D 

— 

— 

- 

- 

— 

- 

— 

— 

>  12D 

- 

- 

- 

- 

- 

- 

- 

( 

Clutter 

Scores 

Pal 

Pfe 

1  By  Mass  [ 

By  Depthb 

All 

Mass 

Oto 

0.25  kg 

>0.25  to 

1  kg 

>1  to 

10  kg 

>  10  kg 

All 

Mass 

Oto 

0.25  kg 

>0.25  to 

1  kg 

>1  to 

8  kg 

<  10kg 

All  Depth 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

— 

- 

— 

- 

- 

— 

— 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

>  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alarm  Rates 

BARres:  |  BARdisc: 

Groups 

Found 

- 

— 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6e.  WOODED  TEST  AREA  RESULTS  (not  covered) 


]  Response  Stage 

Discrimination  Stage 

Munitions" 

Scores 

1  Pdes:  by  type  | 

Pf‘sc:  by  type 

All  Types 

Small 

Medium 

Large 

All  Types 

Small 

Medium 

Large 

- 

- 

- 

- 

- 

- 

- 

- 

I  By  Depthb  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

— 

— 

- 

— 

- 

— 

8D  to  12D 

- 

- 

— 

— 

— 

— 

— 

— 

>  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

Pci 

Pfe 

1  By  Mass  | 

All 

Mass 

SISaM 

>0.25  to 

1  kg 

>1  to 

10  kg 

>10  kg 

All 

Mass 

>0.25  to 

1  kg 

>1  to 

8  kg 

<  10kg 

All  Depth 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

- 

— 

- 

— 

— 

— 

- 

— 

— 

— 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

>  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alann  Rates 

BARres:  |  BARdlsc: 

Groups 

Found 

— 

- 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6f.  MOGUL  TEST  AREA  RESULTS  (not  covered) 


|  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pj™:  by  type  | 

PJ"'C:  by  type 

All  Types 

Small 

Medium 

Large 

All  Types 

Small 

Medium 

Large 

- 

- 

- 

- 

- 

- 

- 

- 

|  By  Dept hh  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

- 

— 

— 

— 

— 

— 

8D  to  12D 

— 

— 

- 

- 

— 

- 

— 

— 

>  12D 

- 

- 

- 

- 

- 

- 

- 

( 

Clutter 

Scores 

Pal 

Pfe 

1  By  Mass  | 

By  Depthb 

All 

Mass 

Oto 

0.25  kg 

>0.25  to 

1  kg 

>1  to 

10  kg 

>  10  kg 

All 

Mass 

Oto 

0.25  kg 

>0.25  to 

1  kg 

>1  to 

8  kg 

<  10kg 

All  Depth 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

— 

- 

— 

- 

- 

— 

— 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

>  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alarm  Rates 

BARres:  |  BARdisc: 

Groups 

Found 

- 

— 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
hAll  depths  are  measured  to  the  center  of  the  object. 


4.3  EFFICIENCY,  REJECTION  RATES,  AND  TYPE  CLASSIFICATION 

Efficiency  and  rejection  rates  are  calculated  to  quantify  the  discrimination  ability  at 
specific  points  of  interest  on  the  ROC  curve:  (1)  at  the  point  where  no  decrease  in  Pd  is  suffered 
(i.e.,  the  efficiency  is  by  definition  equal  to  one)  and  (2)  at  the  operator  selected  threshold. 
These  values  are  presented  in  Tables  7a  through  7d. 
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TABLE  7a.  BLIND  GRID  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

0.98 

0.99 

0.90 

With  No  Loss  of  Pd 

1.00 

0.61 

0.28 

TABLE  7b.  OPEN  FIELD  (DIRECT)  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

0.97 

0.92 

0.72 

With  No  Loss  of  Pd 

1.00 

0.23 

0.06 

TABLE  7c.  OPEN  FIELD  (INDIRECT)  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

0.96 

0.94 

0.85 

With  No  Loss  of  Pd 

1.00 

0.14 

0.05 

TABLE  7d.  OPEN  FIELD  (LEGACY)  EFFICIENCY  AND 
REJECTION  RATES  (not  covered) 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

- 

- 

-- 

TABLE  7e.  WOODED  EFFICIENCY  AND  REJECTION  RATES  (not  covered) 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

-- 

- 

-- 
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TABLE  7f.  MOGUL  EFFICIENCY  AND  REJECTION  RATES  (not  covered) 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

- 

- 

- 

At  the  demonstrator’s  recommended  setting,  the  munitions  items  that  were  detected  and 
correctly  discriminated  were  further  scored  on  whether  their  correct  type  could  be  identified 
(table  8a  through  8f).  Correct  type  examples  include  20-mtn  projectile,  105-mm  HEAT 
projectile,  and  2.75-inch  Rocket.  A  list  of  the  standard  type  declaration  required  for  each 
munitions  item  was  provided  to  demonstrators  prior  to  testing.  The  standard  types  for  the  three 
example  items  are  20-mmP,  105H,  and  2.75-inch. 


TABLE  8a.  BLIND  GRID  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS 


Size 

Percentage  Correct 

25mm 

100% 

37mm 

100% 

60mm 

100% 

81mm 

67% 

105mm 

7% 

105  artillery 

93% 

Overall 

78% 

Note:  The  demonstrator  did  not  attempt  to  provide  type  classification  (if  applicable). 


TABLE  8b.  OPEN  FIELD  DIRECT  FIRE 
CORRECT  TYPE  CLASSIFICATION 
OF  TARGETS  CORRECTLY 
DISCRIMINATED  AS 
MUNITIONS 


Size 

Percentage  Correct 

60mm 

84% 

81mm 

74% 

105mm 

78% 

Overall 

79% 
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TABLE  8c.  OPEN  FIELD  INDIRECT  FIRE 
CORRECT  TYPE  CLASSIFICATION 
OF  TARGETS  CORRECTLY 
DISCRIMINATED  AS 
MUNITIONS 


Size 

Percentage  Correct 

25-mm 

89% 

37-mm 

74% 

105-mm 

91% 

Overall 

85% 

TABLE  8d.  OPEN  FIELD  LEGACY  CORRECT 
TYPE  CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS  (not  covered) 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 

TABLE  8e.  WOODED  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS  (not  covered) 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 
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TABLE  8f.  MOGUL  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS  (not  covered) 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 

4.4  LOCATION  ACCURACY 

The  mean  location  error  and  standard  deviations  appear  in  Tables  9a  through  9f.  These 
calculations  are  based  on  average  missed  distance  for  munitions  correctly  identified  during  the 
response  stage.  Depths  are  measured  from  the  center  of  the  munitions  to  the  surface.  For  the 
blind  grid,  only  depth  errors  are  calculated  because  (X,  Y)  positions  are  known  to  be  the  centers 
of  the  grid  square. 


TABLE  9a.  BLIND  GRID  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

N/A 

N/A 

Easting 

N/A 

N/A 

Depth 

0.05 

0.06 

TABLE  9b.  OPEN  FIELD  DIRECT  FIRE  MEAN 
LOCATION  ERROR  AND 
STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

0.01 

0.08 

Easting 

-0.01 

0.07 

Depth 

0.03 

0.05 
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TABLE  9c.  OPEN  FIELD  INDIRECT  FIRE  MEAN  LOCATION 
ERROR  AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

0.00 

0.07 

Easting 

0.01 

0.07 

Depth 

0.03 

0.06 

TABLE  9d.  OPEN  FIELD  LEGACY  MEAN  LOCATION 
ERROR  AND  STANDARD  DEVIATION  (not  covered) 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

- 

- 

TABLE  9e.  WOODED  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION  (not  covered) 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

- 

- 

-- 

TABLE  9f.  MOGUL  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION  (not  covered) 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

- 

- 

- 

- 
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SECTION  5.  ON-SITE  LABOR  REQUIREMENTS 


A  standardized  estimate  for  labor  associated  with  this  effort  was  calculated  as  follows:  the 
first  person  at  the  test  site  was  designated  supervisor,  the  second  person  was  designated  data 
analyst,  and  the  third  and  following  personnel  were  considered  field  support. 

Government  representatives  monitored  on-site  activity.  All  on-site  activities  were 
grouped  into  one  of  ten  categories:  initial  setup/mobilization,  daily  setup/stop,  calibration, 
collecting  data,  downtime  due  to  break/lunch,  downtime  due  to  equipment  failure,  downtime  due 
to  equipment/data  checks  or  maintenance,  downtime  due  to  weather,  downtime  due  to 
demonstration  site  issue,  or  demobilization.  The  daily  activity  log  is  provided  in  Appendix  D.  A 
summary  of  field  activities  is  provided  in  Section  3.4. 

The  standardized  estimate  of  the  labor  needed  to  perform  the  field  activities  is  presented  in 
Table  10.  Note  that  calibration  time  includes  time  spent  in  the  calibration  lanes  as  well  as  field 
calibrations.  Site  survey  includes  daily  setup/stop  time,  collecting  data,  breaks/lunch,  downtime 
due  to  equipment/data  checks  or  maintenance,  downtime  due  to  failure,  and  downtime  due  to 
weather. 


TABLE  10.  ON-SITE  LABOR  REQUIREMENTS 


No.  of  People 

Hours 

Initial  setup 

Supervisor 

1 

2.75 

Data  analyst 

1 

2.75 

Field  support 

1 

2.75 

Subtotal 

Calibration  site  survey 

Supervisor 

1 

18.92 

Data  analyst 

1 

18.92 

Field  support 

1 

18.92 

Subtotal 

Blind  grid  site  survey 

Supervisor 

1 

17.75 

Data  analyst 

1 

17.75 

Field  support 

1 

17.75 

Subtotal 

See  notes  at  end  of  table. 
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TABLE  10.  (CONT’D) 


No.  of  People 

Hours 

Open  field  site  survey 

Supervisor 

1 

94.33 

Data  analyst 

1 

94.33 

Field  support 

1 

94.33 

Subtotal 

Wooded  site  survey 

Supervisor 

0 

0.00 

Data  analyst 

0 

0.00 

Field  support 

0 

0.00 

Subtotal 

0 

0.00 

Mogul  site  survey 

Supervisor 

0 

0.00 

Data  analyst 

0 

0.00 

Field  support 

0 

0.00 

Subtotal 

0 

0.00 

Demobilization 

Supervisor 

1 

0.92 

Data  analyst 

1 

0.92 

Field  support 

1 

0.92 

Subtotal 

Notes:  Calibration  time  includes  time  spent  in  the  calibration  lanes  as  well  as  calibration 
before  each  data  run. 

Site  survey  time  includes  daily  setup/stop  time,  collecting  data,  breaks/lunch,  downtime 
due  to  system  maintenance,  failure,  and  weather. 
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SECTION  6.  APPENDIXES 


APPENDIX  A.  TERMS  AND  DEFINITIONS 

GENERAL  DEFINITIONS 

Anomaly:  Location  of  a  system  response  deemed  to  warrant  further  investigation  by  the 
demonstrator  for  consideration  as  an  emplaced  munitions  item. 

Detection:  An  anomaly  location  that  is  within  Rhai0  of  an  emplaced  munitions  item. 

Military  Munitions  (MM):  Specific  categories  of  MM  that  may  pose  unique  explosive  safety 
risks,  including  UXO  as  defined  in  10USC  101(e)(5),  DMM  as  defined  in  10  USC  2710(e)(2) 
and/or  munitions  constituents  (e.g.  TNT,  RDX)  as  defined  in  10  USC  2710(e)(3)  that  are  present 
in  high  enough  concentrations  to  pose  an  explosive  hazard. 

Emplaced  Munitions:  A  munitions  item  buried  by  the  government  at  a  specified  location  in  the 
test  site. 

Emplaced  Clutter:  A  clutter  item  (i.e.,  nonmunitions  item)  buried  by  the  government  at  a 
specified  location  in  the  test  site. 

Rhaio:  A  predetermined  radius  about  an  emplaced  item  (clutter  or  munitions)  within  which  an 
anomaly  identified  by  the  demonstrator  as  being  of  interest  is  considered  to  be  a  detection  of  that 
item.  For  the  purpose  of  this  program,  a  circular  halo  0.5  meters  in  radius  is  placed  around  the 
center  of  the  object  for  all  clutter  and  munitions  items. 

Small  Munitions:  Caliber  of  munitions  less  than  or  equal  to  40  mm  (includes  20-mm  projectile, 
25-mm  projectile,  37-mm  projectile,  40-mm  projectile,  submunitions  BLU-26,  BLU-63,  and 
M42). 

Medium  Munitions:  Caliber  of  munitions  greater  than  40  mm  and  less  than  or  equal  to  8 1  mm 
(includes  57-mm  projectile,  60-mm  mortar,  2.75-inch  rocket,  and  81-mm  mortar). 

Large  Munitions:  Caliber  of  munitions  greater  than  81  mm  (includes  105-mm  HEAT,  105-mm 
projectile,  and  155-mm  projectile). 

Group:  Two  or  more  adjacent  GT  items  with  overlapping  halos. 

GT:  Ground  truth 

Response  Stage  Noise  Level:  The  level  that  represents  the  signal  level  below  which  anomalies 
are  not  considered  detectable.  Demonstrators  are  required  to  provide  the  recommended  noise 
level  for  the  blind  grid  test  area. 


A-l 


Discrimination  Stage  Threshold:  The  demonstrator-selected  threshold  level  that  is  expected  to 
provide  optimum  performance  of  the  system  by  retaining  all  detectable  munitions  and  rejecting 
the  maximum  amount  of  clutter.  This  level  defines  the  subset  of  anomalies  the  demonstrator 
would  recommend  digging  based  on  discrimination. 

Binomially  Distributed  Random  Variable:  A  random  variable  of  the  type  which  has  only  two 
possible  outcomes,  say  success  and  failure,  is  repeated  for  n  independent  trials  with  the 
probability  p  of  success  and  the  probability  1-p  of  failure  being  the  same  for  each  trial.  The 
number  of  successes  x  observed  in  the  n  trials  is  an  estimate  of  p  and  is  considered  to  be  a 
binomially  distributed  random  variable. 

RESPONSE  AND  DISCRIMINATION  STAGE  DATA 

The  scoring  of  the  demonstrator’s  performance  is  conducted  in  two  stages:  response  stage 
and  discrimination  stage.  For  both  stages,  the  probability  of  detection  (Pd)  and  the  false  alarms 
are  reported  as  receiver-operating  characteristic  (ROC)  curves.  False  alanns  are  divided  into 
those  anomalies  that  correspond  to  emplaced  clutter  items,  measuring  the  probability  of  clutter 
detection  (Pcd)  or  probability  of  false  positive  (Pfp).  Those  that  do  not  correspond  to  any  known 
item  are  termed  background  alanns. 

The  response  stage  is  a  measure  of  whether  the  sensor  can  detect  an  object  of  interest.  For 
a  channel  instrument,  this  value  should  be  closely  related  to  the  amplitude  of  the  signal.  The 
demonstrator  must  report  the  response  level  (threshold)  below  which  target  responses  are 
deemed  insufficient  to  warrant  further  investigation.  At  this  stage,  minimal  processing  may  be 
done.  This  includes  filtering  long-  and  short-scale  variations,  bias  removal,  and  scaling.  This 
processing  should  be  detailed  in  the  data  submission. 

For  a  multichannel  instrument,  the  demonstrator  must  construct  a  quantity  analogous  to 
amplitude.  The  demonstrator  should  consider  what  combination  of  channels  provides  the  best 
test  for  detecting  any  object  that  the  sensor  can  detect.  The  average  amplitude  across  a  set  of 
channels  is  an  example  of  an  acceptable  response  stage  quantity.  Other  methods  may  be  more 
appropriate  for  a  given  sensor.  Again,  minimal  processing  can  be  done,  and  the  demonstrator 
should  explain  how  this  quantity  was  constructed  in  their  data  submission. 

The  discrimination  stage  evaluates  the  demonstrator’s  ability  to  correctly  identify 
munitions  as  such,  and  to  reject  clutter.  For  the  same  locations  as  in  the  response  stage  anomaly 
list,  the  discrimination  stage  list  contains  the  output  of  the  algorithms  applied  in  the 
discrimination-stage  processing.  This  list  is  prioritized  based  on  the  demonstrator’s 
determination  that  an  anomaly  location  is  likely  to  contain  munitions.  Thus,  higher  output  values 
are  indicative  of  higher  confidence  that  a  munitions  item  is  present  at  the  specified  location.  For 
electronic  signal  processing,  priority  ranking  is  based  on  algorithm  output.  For  other  systems, 
priority  ranking  is  based  on  human  judgment.  The  demonstrator  also  selects  the  threshold  that 
the  demonstrator  believes  will  provide  optimum  system  performance,  (i.e.,  that  retains  all  the 
detected  munitions  and  rejects  the  maximum  amount  of  clutter). 
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Note:  The  two  lists  provided  by  the  demonstrator  contain  identical  numbers  of  potential  target 
locations.  They  differ  only  in  the  priority  ranking  of  the  declarations. 

GROUP  SCORING  FACTORS 

Based  on  configuration  of  the  GT  at  the  standardized  sites  and  the  defined  scoring 
methodology,  there  exists  munitions  groups  defined  as  having  overlapping  halos.  In  these  cases, 
the  following  scoring  logic  is  implemented  (fig.  A-l  through  A-9): 

a.  Overall  site  scores  (i.e.,  Pd)  will  consider  only  isolated  munitions  and  clutter  items. 

b.  GT  items  that  have  overlapping  halos  (both  munitions  and  clutter)  will  form  a  group 
and  groups  may  form  chains. 

c.  Groups  will  have  a  complex  halos  composed  of  all  the  composite  halos  of  all  its  GT 

items. 

d.  Groups  will  have  three  scoring  factors:  groups  found  groups  identified  and  group 
coverage.  Scores  will  be  based  on  1 : 1  matches  of  anomalies  and  GT. 

(1)  Groups  Found  (Found):  the  number  of  groups  that  have  one  or  more  GT 
items  matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with 
detecting  a  group  if  any  item  within  the  group  is  matched  to  an  anomaly  in  their  list. 

(2)  Groups  Identified  (ID):  the  number  of  groups  that  have  two  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  identifying 
that  a  group  is  present  if  multiple  items  within  the  composite  halo  are  matched  to  anomalies  in 
their  list. 

(3)  Group  Coverage  (Coverage):  the  number  of  GT  items  matched  within  groups  divided 
by  the  total  number  of  GT  items  within  groups.  This  metric  measures  the  demonstrator  accuracy 
in  detennining  the  number  of  anomalies  within  a  group.  If  five  items  are  present  and  only  two 
anomalies  are  matched,  the  demonstrator  will  score  0.4.  If  all  five  are  matched  the  demonstrator 
will  score  1.0. 

e.  Location  error  will  not  be  reported  for  groups. 
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f.  Demonstrators  will  not  be  asked  to  call  out  groups  in  their  scoring  submissions.  If 
multiple  anomalies  are  indicated  in  a  small  area,  the  demonstrator  will  report  all  individual 
anomalies. 

g.  Excess  alarms  within  a  halo  will  be  disregarded. 


A- 1 .  Example  of  detected  item. 
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A-3.  Example  of  group  identified  (ID). 


A-4.  Example  of  excess  alarms  disregarded. 
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A-6.  Example  of  group  (1/4  =  0.25). 
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A-7.  Example  of  group  (2/4  =  0.5). 
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RESPONSE  STAGE  DEFINITIONS 

Response  Stage  Probability  of  Detection  (Pdres):  Pdres  =  (No.  of  response-stage  detections)/ 
(No.  of  emplaced  munitions  in  the  test  site). 

Response  Stage  Clutter  Detection  (cdies):  An  anomaly  location  that  is  within  Rhai0  of  an 
emplaced  clutter  item. 

Response  Stage  Probability  of  Clutter  Detection  (Pcdres):  Pcdres  =  (No.  of  response-stage  clutter 
detections)/(No.  of  emplaced  clutter  items). 

Response  Stage  Background  Alarm  (bares):  An  anomaly  in  a  blind  grid  cell  that  contains  neither 
emplaced  munitions  nor  an  emplaced  clutter  item.  An  anomaly  location  in  the  open  field  or 
scenarios  that  is  outside  Rhaio  of  any  emplaced  munitions  or  emplaced  clutter  item. 

Response  Stage  Probability  of  Background  Alarm  (Pb/es):  Blind  grid  only:  Pbares  =  (No.  of 
response-stage  background  alarms)/(No.  of  empty  grid  locations). 

Response  Stage  Background  Alarm  Rate  (BARres):  Open  field  any  challenge  area  (including  the 
direct  and  indirect  firing  sub  areas)  only:  BARres  =  (No.  of  response-stage  background 
alarms)/(arbitrary  constant). 

Note  that  the  quantities  Pc|'es,  Pcdres,  Pbares,  and  BARres  are  functions  of  tres,  the  threshold 
applied  to  the  response-stage  signal  strength.  These  quantities  can  therefore  be  written  as 

Pdres(tres),  Pcdres(tres),  Pbares(tres),  and  BARres(tres). 
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DISCRIMINATION  STAGE  DEFINITIONS 


Discrimination:  The  application  of  a  signal  processing  algorithm  or  human  judgment  to  sensor 
data  to  discriminate  munitions  from  clutter.  Discrimination  should  identify  anomalies  that  the 
demonstrator  has  high  confidence  correspond  to  munitions,  as  well  as  those  that  the  demonstrator 
has  high  confidence  correspond  to  nonmunitions  or  background  returns.  The  former  should  be 
ranked  with  highest  priority  and  the  latter  with  lowest. 

Discrimination  Stage  Probability  of  Detection  (Pddlsc):  Pddlsc  =  (No.  of  discrimination-stage 
detections)/(No.  of  emplaced  munitions  in  the  test  site). 

Discrimination  Stage  False  Positive  (fpdlsc):  An  anomaly  location  that  is  within  Rhaio  of  an 
emplaced  clutter  item. 

Discrimination  Stage  Probability  of  False  Positive  (Pfpdlsc):  Pfpdlsc  =  (No.  of  discrimination  stage 
false  positives)/(No.  of  emplaced  clutter  items). 

Discrimination  Stage  Background  Alarm  (badlsc):  An  anomaly  in  a  blind  grid  cell  that  contains 
neither  emplaced  munitions  nor  an  emplaced  clutter  item.  An  anomaly  location  in  the  open  field 
or  scenarios  that  is  outside  Rhaio  of  any  emplaced  munitions  or  emplaced  clutter  item. 

Discrimination  Stage  Probability  of  Background  Alarm  (Pbadlsc):  Pbadlsc  =  (No.  of  discrimination- 
stage  background  alarms )/(No.  of  empty  grid  locations). 

Discrimination  Stage  Background  Alann  Rate  (BARdlsc):  BARdlsc  =  (No.  of  discrimination-stage 
background  alanns)/(arbitrary  constant). 

Note  that  the  quantities  Pddlsc,  Pfpdisc,  Pbadisc,  and  BARdlsc  are  functions  of  tdlsc,  the  threshold 
applied  to  the  discrimination-stage  signal  strength.  These  quantities  can  therefore  be  written  as 
Pdd'sc(f,lsc),  Pfpdlsc(tdisc),  Pbadisc(tdisc),  and  BARdisc(tdisc). 
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RECEIVER-OPERATING  CHARACERISTIC  (ROC)  CURVES 


ROC  curves  at  both  the  response  and  discrimination  stages  can  be  constructed  based  on  the 
above  definitions.  The  ROC  curves  plot  the  relationship  between  Pd  versus  Pcd  or  Pfp  and  Pd 
versus  BAR  or  Pba  as  the  threshold  applied  to  the  signal  strength  is  varied  from  its  minimum 
(tmin)  to  its  maximum  (tmax)  value. 1  Pd  versus  Pfp  and  Pd  versus  BAR  being  combined  into  ROC 
curves  is  shown  in  Figure  A- 10.  Note  that  the  “res”  and  “disc”  superscripts  have  been 
suppressed  from  all  the  variables  for  clarity. 


Figure  A- 10.  ROC  curves  for  open  field  testing.  Each  curve  applies  to  both  the  response  and 
discrimination  stages. 


METRICS  TO  CHARACTERIZE  THE  DISCRIMINATION  STAGE 

The  demonstrator  is  also  scored  on  efficiency  and  rejection  ratio,  which  measure  the 
effectiveness  of  the  discrimination  stage  processing.  The  goal  of  discrimination  is  to  retain  the 
greatest  number  of  munitions  detections  from  the  anomaly  list  while  rejecting  the  maximum 
number  of  anomalies  arising  from  nonmunitions  items.  The  efficiency  measures  the  fraction  of 
detected  munitions  retained  by  the  discrimination,  while  the  rejection  ratio  measures  the  fraction 
of  false  alarms  rejected.  Both  measures  are  defined  relative  to  the  entire  response  list,  i.e.,  the 
maximum  munitions  detectable  by  the  sensor  and  its  accompanying  clutter  detection  rate/false 
positive  rate  or  background  alarm  rate. 


'Strictly  speaking,  ROC  curves  plot  the  Pd  versus  Pba  over  a  predetermined  and  fixed  number  of 
detection  opportunities  (some  of  the  opportunities  are  located  over  munitions  and  others  are 
located  over  clutter  or  blank  spots).  In  an  open  field  scenario,  each  system  suppresses  its  signal 
strength  reports  until  some  bare-minimum  signal  response  is  received  by  the  system. 
Consequently,  the  open  field  ROC  curves  do  not  have  information  from  low  signal-output 
locations,  and,  furthermore,  different  contractors  report  their  signals  over  a  different  set  of 
locations  on  the  ground.  These  ROC  curves  are  thus  not  true  to  the  strict  definition  of  ROC 
curves  as  defined  in  textbooks  on  detection  theory.  Note,  however,  that  the  ROC  curves 
obtained  in  the  blind  grid  test  sites  are  true  ROC  curves. 
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Efficiency  (E):  E  =  Pddlsc(tdlsc)/Pdres(tminres):  Measures  (at  a  threshold  of  interest)  the  degree 
to  which  the  maximum  theoretical  detection  performance  of  the  sensor  system  (as  detennined  by 
the  response  stage  tmin)  is  preserved  after  application  of  discrimination  techniques.  Efficiency  is 
a  number  between  0  and  1 .  An  efficiency  of  1  implies  that  all  of  the  munitions  initially  detected 
in  the  response  stage  were  retained  at  the  specified  threshold  in  the  discrimination  stage,  tdlsc. 

False  Positive  Rejection  Rate  (Rfp):  Rrp  =  1  -  [Pfpdisc(tdisc)/Pcdres(tminres)]:  Measures  (at  a 
threshold  of  interest)  the  degree  to  which  the  sensor  system's  false  positive  performance  is 
improved  over  the  maximum  false  positive  performance  (as  detennined  by  the  response  stage 
tmin).  The  rejection  rate  is  a  number  between  0  and  1.  A  rejection  rate  of  1  implies  that  all 
emplaced  clutter  initially  detected  in  the  response  stage  were  correctly  rejected  at  the  specified 
threshold  in  the  discrimination  stage. 

Background  Alann  Rejection  Rate  (Rba): 

Blind  grid:  Rba  =  1  -  [Pbadisc(tdisc)/Pbares(tminres)]  - 

Open  field:  Rba=  1  -  [BARdlsc(tdlsc)/BARres(tminres)]). 

Measures  the  degree  to  which  the  discrimination  stage  correctly  rejects  background  alarms 
initially  detected  in  the  response  stage.  The  rejection  rate  is  a  number  between  0  and  1.  A 
rejection  rate  of  1  implies  that  all  background  alanns  initially  detected  in  the  response  stage  were 
rejected  at  the  specified  threshold  in  the  discrimination  stage. 

CHI-SQUARE  COMPARISON 

The  Chi-square  test  for  differences  in  probabilities  (or  2  by  2  contingency  table)  is  used  to 
analyze  two  samples  drawn  from  two  different  populations  to  see  if  both  populations  have  the 
same  or  different  proportions  of  elements  in  a  certain  category.  More  specifically,  two  random 
samples  are  drawn,  one  from  each  population,  to  test  the  null  hypothesis  that  the  probability  of 
event  A  (some  specified  event)  is  the  same  for  both  populations  (ref  3). 

The  test  statistic  of  the  2  by  2  contingency  table  is  the  Chi-square  distribution  with  one 
degree  of  freedom.  When  an  association  between  a  more  challenging  terrain  feature  and 
relatively  degraded  perfonnance  is  sought,  a  one-sided  test  is  performed.  A  two-sided  2  by  2 
contingency  table  is  used  in  the  Standardized  UXO  Technology  Demonstration  Site  Program  to 
compare  perfonnance  between  any  two  areas  or  subareas  when  the  direction  of  degradation 
cannot  be  predetermined. 

For  a  one-sided  test,  a  significance  level  of  0.05  is  used  to  set  the  critical  decision  limit.  It 
is  a  critical  decision  limit  because  if  the  test  statistic  calculated  from  the  data  exceeds  this  value, 
then  the  lower  proportion  tested  will  be  considered  significantly  less  than  the  greater  one 
(degraded).  If  the  test  statistic  calculated  from  the  data  is  less  than  this  value,  then  no 
degradation  can  be  said  to  exist  because  of  the  terrain  feature  introduced. 
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For  a  two-sided  test,  a  significance  level  of  0.10  is  used  to  allow  .05  on  either  side  of  the 
decision.  It  is  a  critical  decision  limit  because  if  the  test  statistic  calculated  from  the  data 
exceeds  this  value,  then  the  two  proportions  tested  will  be  considered  significantly  different.  If 
the  test  statistic  calculated  from  the  data  is  less  than  this  value,  then  the  two  proportions  tested 
will  be  considered  not  significantly  different. 

An  exception  must  be  applied  when  either  a  0  or  100  percent  success  rate  occurs  in  the 
sample  data.  The  Chi-square  test  cannot  be  used  in  these  instances.  Instead,  Fischer’s  test  is 
used,  and  the  critical  decision  limit  for  one-sided  tests  is  the  chosen  significance  level,  which  in 
this  case  is  0.05.  With  Fischer’s  test,  if  the  test  statistic  is  less  than  the  critical  value,  then  the 
proportions  are  considered  to  be  significantly  different. 

An  example  follows  that  illustrates  Standardized  UXO  Technology  Demonstration  Site 
blind  grid  results  compared  to  those  from  the  open  field  legacy.  It  should  be  noted  that  a 
significant  result  does  not  prove  a  cause-and-effect  relationship  exists  between  the  two 
populations  of  interest;  however,  it  does  serve  as  a  tool  to  indicate  that  one  data  set  has 
experienced  a  degradation  or  change  in  system  perfonnance  at  a  large  enough  level  than  can  be 
accounted  for  merely  by  chance  or  random  variation.  Note  also  that  a  result  that  is  not 
significant  indicates  that  there  is  not  enough  evidence  to  declare  that  anything  more  than  chance 
or  random  variation  within  the  same  population  is  at  work  between  the  two  data  sets  being 
compared. 

Demonstrator  X  achieves  the  following  overall  results  after  surveying  the  blind  grid  and 
open  field  (legacy)  using  the  same  system  (results  indicate  the  number  of  munitions  detected 
divided  by  the  number  of  munitions  emplaced): 

Blind  grid  Open  field 

Pdres  100/100  =  1.0  8/10  =  .80 


P/es:  BLIND  GRID  versus  OPEN  FIELD  (legacy).  Using  the  example  data  above  to 
compare  probabilities  of  detection  in  the  response  stage,  all  100  munitions  out  of  100  emplaced 
munitions  items  were  detected  in  the  blind  grid  while  8  munitions  out  of  10  emplaced  were 
detected  in  the  open  field.  Fischer’s  test  must  be  used  since  a  100  percent  success  rate  occurs  in 
the  data.  Fischer’s  test  uses  the  four  input  values  to  calculate  a  test  statistic  of  0.0075  that  is 
compared  against  the  critical  value  of  0.05.  Since  the  test  statistic  is  less  than  the  critical  value, 
the  smaller  response  stage  detection  rate  (0.80)  is  considered  to  be  significantly  less  at  the  0.05 
level  of  significance.  While  a  significant  result  does  not  prove  a  cause-and-effect  relationship 
exists  between  the  change  in  survey  area  and  degradation  in  performance,  it  does  indicate  that 
the  detection  ability  of  demonstrator  X’s  system  seems  to  have  been  degraded  in  the  open  field 
relative  to  results  from  the  blind  grid  using  the  same  system.  This  is  an  example  of  a  one-sided 
Chi-squared  test. 
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APPENDIX  B.  DAILY  WEATHER  LOGS 


Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

15  Sep 

0100 

81.1 

0.00  | 

0200 

82.0 

0.00 

0300 

80.8 

0.00 

0400 

79.5 

0.00 

0500 

77.5 

0.00 

0600 

77.4 

0.00 

0700 

76.6 

0.00  ! 

0800 

77.7 

0.00 

0900 

79.3 

0.00 

1000 

80.4 

0.00 

1100 

81.1 

0.00 

1200 

81.3 

0.00 

1300 

82.2 

0.00 

1400 

82.2 

0.00 

1500 

82.6 

0.00 

1600 

82.6 

0.00 

1700 

81.0 

0.00 

1800 

77.9 

0.00 

1900 

74.5 

0.00 

2000 

72.3 

0.00 

2100 

72.3 

0.00 

2200 

72.1 

0.00 

2300 

71.6 

0.00 

2359 

70.7 

0.00 

16  Sep 

0100 

69.1 

0.00 

0200 

67.3 

0.00 

0300 

65.5 

0.00 

0400 

63.7 

0.00 

0500 

63.0 

0.00  j 

0600 

62.6 

0.00 

0700 

62.8 

o.oo  : 

0800 

63.9 

0.00 

0900 

65.3 

0.00 

1000 

66.2 

0.00 

1100 

67.8 

0.00 

1200 

69.1 

0.00 

1300 

70.3 

0.00 

1400 

72.3 

0.00 

1500 

72.5 

0.00 

1600 

71.4 

0.00 

1700 

70.7 

0.00 

1800 

69.1 

0.00 

1900 

66.2 

0.00 
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Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

16  Sep 

2000 

62.6 

0.00 

2100 

60.3 

0.00 

2200 

58.5 

0.00 

2300 

57.2 

0.00 

2359 

56.3 

0.00 

17  Sep 

0100 

55.4 

0.00 

0200 

55.9 

0.00 

0300 

56.8 

0.00 

0400 

54.7 

0.00 

0500 

54.0 

0.00 

0600 

53.8 

0.00 

0700 

56.8 

0.00 

0800 

63.1 

0.00 

0900 

64.8 

0.00 

1000 

66.9 

0.00 

1100 

68.2 

0.00 

1200 

70.7 

0.00 

1300 

72.0 

0.00 

1400 

73.9 

0.00 

1500 

75.0 

0.00 

1600 

75.7 

0.00 

1700 

74.7 

0.00 

1800 

71.2 

0.00 

1900 

66.6 

0.00 

2000 

63.0 

0.00 

2100 

60.3 

0.00 

2200 

58.5 

0.00 

2300 

57.6 

0.00 

2359 

57.2 

0.00 

18  Sep 

0100 

56.8 

0.00 

0200 

56.1 

0.00 

0300 

55.0 

0.00 

0400 

54.3 

0.00 

0500 

53.8 

0.00 

0600 

54.0 

0.00 

0700 

55.4 

0.00 

0800 

64.2 

0.00 

0900 

69.4 

0.00 

1000 

72.1 

0.00 

1100 

74.5 

0.00 

1200 

76.1 

0.00 

1300 

76.8 

0.00 

1400 

77.0 

0.00 

1500 

77.5 

0.00 

1600 

76.6 

0.00 

1700 

75.4 

0.00 
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Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

18  Sep 

1800 

73.4 

0.00 

1900 

69.3 

0.00 

2000 

67.3 

0.00 

2100 

68.7 

0.00 

2200 

67.3 

0.00 

2300 

66.4 

0.00 

2359 

64.6 

0.00 

19  Sep 

0100 

64.0 

0.00 

0200 

62.2 

0.00 

0300 

60.8 

0.00 

0400 

60.1 

0.00 

0500 

59.2 

0.00 

0600 

58.1 

0.00 

0700 

58.5 

0.00 

0800 

61.3 

0.00 

0900 

63.1 

0.00 

1000 

65.7 

0.00 

1100 

67.5 

0.00 

1200 

68.5 

0.00 

1300 

69.4 

0.00 

1400 

70.0 

0.00 

1500 

70.2 

0.00 

1600 

70.3 

0.00 

1700 

68.2 

0.00 

1800 

66.9 

0.00 

1900 

63.3 

0.00 

2000 

61.9 

0.00 

2100 

61.2 

0.00 

2200 

59.9 

0.00 

2300 

58.3 

0.00 

2359 

57.4 

0.00 

20  Sep 

0100 

55.4 

0.00 

0200 

53.4 

0.00 

0300 

51.8 

0.00 

0400 

51.3 

0.00 

0500 

50.4 

0.00 

0600 

50.0 

0.00 

0700 

50.5 

0.00 

0800 

56.1 

0.00 

0900 

61.3 

0.00 

1000 

64.6 

0.00 

1100 

66.6 

0.00 

1200 

67.5 

0.00 

1300 

68.9 

0.00 

1400 

69.4 

0.00 

1500 

70.7 

0.00 

B-3 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

20  Sep 

1600 

71.2 

0.00 

1700 

69.4 

0.00 

1800 

67.3 

0.00 

1900 

62.4 

0.00 

2000 

59.5 

0.00 

2100 

57.7 

0.00 

2200 

55.6 

0.00 

2300 

54.1 

0.00 

2359 

53.2 

0.00 

21  Sep 

0100 

52.5 

0.00 

0200 

51.4 

0.00 

0300 

50.2 

0.00 

0400 

49.5 

0.00 

0500 

48.9 

0.00 

0600 

49.1 

0.00 

0700 

50.4 

0.00 

0800 

58.1 

0.00 

0900 

64.4 

0.00 

1000 

69.6 

0.00 

1100 

72.9 

0.00 

1200 

75.6 

0.00 

1300 

77.2 

0.00 

1400 

78.8 

0.00 

1500 

78.3 

0.00 

1600 

78.1 

0.00 

1700 

77.0 

0.00 

1800 

74.1 

0.00 

1900 

67.6 

0.00 

2000 

64.6 

0.00 

2100 

62.1 

0.00 

2200 

60.8 

0.00 

2300 

59.7 

0.00 

2359 

58.5 

0.00 

22  Sep 

0100 

57.6 

0.00 

0200 

57.0 

0.00 

0300 

56.7 

0.00 

0400 

56.1 

0.00 

0500 

59.0 

0.00 

0600 

59.2 

0.00 

0700 

59.7 

0.00 

0800 

63.5 

0.00 

0900 

67.6 

0.00 

1000 

69.8 

0.00 

1100 

72.3 

0.00 

1200 

74.5 

0.00 

B-4 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

22  Sep 

1300 

75.9 

0.00 

1400 

77.0 

0.00 

1500 

77.7 

0.00 

1600 

76.5 

0.00 

1700 

75.7 

0.00 

1800 

74.3 

0.00 

1900 

73.2 

0.00 

2000 

72.1 

0.00 

2100 

71.6 

0.00 

2200 

70.2 

0.00 

2300 

67.8 

0.00 

2359 

65.8 

0.00 

23  Sep 

0100 

65.3 

0.00 

0200 

64.6 

0.00 

0300 

63.7 

0.00 

0400 

61.9 

0.00 

0500 

60.4 

0.00 

0600 

59.4 

0.00 

0700 

59.5 

0.00 

0800 

61.9 

0.00 

0900 

64.0 

0.00 

1000 

66.2 

0.00 

1100 

67.8 

0.00 

1200 

69.1 

0.00 

1300 

70.3 

0.00 

1400 

71.4 

0.00 

1500 

72.1 

0.00 

1600 

72.3 

0.00 

1700 

71.4 

0.00 

1800 

68.9 

0.00 

1900 

64.2 

0.00 

2000 

59.7 

0.00 

2100 

56.8 

0.00 

2200 

57.7 

0.00 

2300 

59.0 

0.00 

2359 

55.4 

0.00 

24  Sep 

0100 

55.0 

0.00 

0200 

55.6 

0.00 

0300 

55.0 

0.00 

0400 

54.7 

0.00 

0500 

53.2 

0.00 

0600 

53.8 

0.00 

0700 

55.8 

0.00 

0800 

60.3 

0.00 

0900 

64.0 

0.00 

1000 

65.5 

0.00 

B-5 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

24  Sep 

1100 

68.0 

0.00 

1200 

69.4 

0.00 

1300 

70.9 

0.00 

1400 

72.0 

0.00 

1500 

71.6 

0.00 

1600 

71.4 

0.00 

1700 

70.5 

0.00 

1800 

68.5 

0.00 

1900 

67.1 

0.00 

2000 

65.7 

0.00 

2100 

64.0 

0.00 

2200 

62.4 

0.00 

2300 

61.7 

0.00 

2359 

60.8 

0.00 

25  Sep 

0100 

59.4 

0.00 

0200 

58.8 

0.00 

0300 

57.9 

0.00 

0400 

57.9 

0.00 

0500 

57.2 

0.00 

0600 

56.5 

0.00 

0700 

56.7 

0.00 

0800 

58.5 

0.00 

0900 

59.9 

0.00 

1000 

61.9 

0.00 

1100 

63.3 

0.00 

1200 

64.8 

0.00 

1300 

64.8 

0.00 

1400 

63.0 

0.02 

1500 

62.6 

0.01 

1600 

63.5 

0.00 

1700 

63.5 

0.01 

1800 

61.9 

0.01 

1900 

61.3 

0.01 

2000 

60.8 

0.03 

2100 

60.6 

0.01 

2200 

59.5 

0.03 

2300 

59.9 

0.00 

2359 

60.4 

0.00 

26  Sep 

0100 

60.6 

0.00 

0200 

61.0 

0.00 

0300 

61.7 

0.00 

0400 

62.2 

0.00 

0500 

62.4 

0.02 

0600 

62.1 

0.01 

0700 

62.4 

0.00 

0800 

63.1 

0.00 

B-6 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

26  Sep 

0900 

64.0 

0.00 

1000 

65.1 

0.00 

1100 

66.4 

0.00 

1200 

67.8 

0.00 

1300 

68.2 

0.00 

1400 

67.6 

0.00 

1500 

67.8 

0.00 

1600 

68.9 

0.00 

1700 

68.9 

0.00 

1800 

68.5 

0.00 

1900 

68.7 

0.00 

2000 

68.9 

0.00 

2100 

68.7 

0.00 

2200 

68.9 

0.01 

2300 

68.7 

0.01 

2359 

68.5 

0.06 

27  Sep 

0100 

68.7 

0.00 

0200 

68.9 

0.00 

0300 

69.1 

0.00 

0400 

69.3 

0.05 

0500 

69.3 

0.01 

0600 

69.4 

0.00 

0700 

70.2 

0.00 

0800 

70.5 

0.05 

0900 

71.2 

0.00 

1000 

72.0 

0.00 

1100 

72.1 

0.15 

1200 

70.3 

0.12 

1300 

71.4 

0.00 

1400 

72.0 

0.01 

1500 

72.7 

0.00 

1600 

73.8 

0.00 

1700 

73.8 

0.00 

1800 

72.3 

0.00 

1900 

72.0 

0.00 

2000 

71.2 

0.00 

2100 

70.5 

0.00 

2200 

70.0 

0.05 

2300 

69.4 

0.02 

2359 

69.4 

0.09 

28  Sep 

0100 

69.3 

0.00 

0200 

69.3 

0.00 

0300 

69.1 

0.00 

0400 

68.9 

0.00 

0500 

68.7 

0.00 

0600 

68.7 

0.00 

B-7 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

28  Sep 

0700 

68.9 

0.00 

0800 

69.6 

0.00 

0900 

70.2 

0.00 

1000 

72.0 

0.00 

1100 

73.4 

0.00 

1200 

76.3 

0.00 

1300 

76.8 

0.01 

1400 

73.9 

0.05 

1500 

74.8 

0.00 

1600 

72.9 

0.10 

1700 

71.2 

0.01 

1800 

70.2 

0.00 

1900 

68.7 

0.00 

2000 

68.7 

0.00 

2100 

68.5 

0.00 

2200 

68.0 

0.00 

2300 

67.5 

0.00 

2359 

67.5 

0.00 

29  Sep 

0100 

66.9 

0.00 

0200 

66.4 

0.00 

0300 

65.8 

0.00 

0400 

64.9 

0.00 

0500 

64.2 

0.00 

0600 

63.0 

0.00 

0700 

63.0 

0.00 

0800 

66.9 

0.00 

0900 

69.8 

0.00 

1000 

71.6 

0.00 

1100 

72.3 

0.00 

1200 

72.0 

0.00 

1300 

72.0 

0.00 

1400 

71.8 

0.00 

1500 

72.7 

0.00 

1600 

73.8 

0.00 

1700 

72.5 

0.00 

1800 

69.6 

0.00 

1900 

66.0 

0.00 

2000 

63.0 

0.00 

2100 

60.8 

0.00 

2200 

59.7 

0.00 

2300 

59.5 

0.00 

2359 

57.7 

0.00 

30  Sep 

0100 

55.6 

0.00 

0200 

54.3 

0.00 

0300 

53.2 

0.00 

0400 

53.1 

0.00 

B-8 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

30  Sep 

0500 

52.3 

0.00 

0600 

51.6 

0.00 

0700 

52.5 

0.00 

0800 

58.6 

0.00 

0900 

63.9 

0.00 

1000 

65.8 

0.00 

1100 

68.4 

0.00 

1200 

71.2 

0.00 

1300 

72.0 

0.00 

1400 

72.7 

0.00 

1500 

72.7 

0.00 

1600 

72.7 

0.00 

1700 

71.2 

0.00 

1800 

69.1 

0.00 

1900 

68.2 

0.14 

2000 

62.2 

0.13 

2100 

61.2 

0.01 

2200 

61.3 

0.05 

2300 

60.8 

0.00 

2359 

60.3 

0.00 

1  Oct 

0100 

60.1 

0.00 

0200 

59.7 

0.01 

0300 

59.2 

0.00 

0400 

58.5 

0.00 

0500 

57.2 

0.00 

0600 

55.4 

0.00 

0700 

54.9 

0.00 

0800 

57.6 

0.00 

0900 

59.9 

0.00 

1000 

63.3 

0.00 

1100 

68.0 

0.00 

1200 

70.9 

0.00 

1300 

72.1 

0.00 

1400 

66.4 

0.17 

1500 

63.3 

0.06 

1600 

62.1 

0.02 

1700 

61.2 

0.00 

1800 

60.4 

0.00 

1900 

59.7 

0.00 

2000 

59.2 

0.01 

2100 

58.3 

0.02 

2200 

57.4 

0.03 

2300 

55.9 

0.00 

2359 

53.6 

0.00 

2  Oct 

0100 

52.5 

0.00 

0200 

51.4 

0.00 

B-9 


Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

0300 

50.4 

0.00 

0400 

50.0 

0.00 

0500 

49.5 

0.00 

0600 

49.6 

0.00 

0700 

50.2 

0.00 

0800 

54.1 

0.00 

0900 

57.0 

0.00 

1000 

59.0 

0.00 

1100 

60.8 

0.00 

1200 

61.2 

0.00 

1300 

62.2 

0.00 

1400 

62.8 

0.00 

1500 

63.0 

0.00 

1600 

62.8 

0.00 

1700 

62.2 

0.00 

1800 

59.0 

0.00 

1900 

55.6 

0.00 

2000 

50.2 

0.00 

2100 

47.8 

0.00 

2200 

47.3 

0.00 

2300 

46.6 

0.00 

2359 

45.5 

0.00 

0100 

45.0 

0.00 

0200 

44.6 

0.00 

0300 

44.8 

0.00 

0400 

46.4 

0.00 

0500 

48.9 

0.00 

0600 

49.5 

0.00 

0700 

50.2 

0.00 

0800 

55.2 

0.00 

0900 

59.4 

0.00 

1000 

63.7 

0.00 

1100 

66.4 

0.00 

1200 

68.7 

0.00 

1300 

69.4 

0.00 

1400 

69.4 

0.00 

1500 

69.6 

0.00 

1600 

69.3 

0.00 

1700 

67.6 

0.00 

1800 

64.6 

0.00 

1900 

60.1 

0.00 

2000 

57.0 

0.00 

2100 

54.1 

0.00 

2200 

53.4 

0.00 

2300 

53.1 

0.00 

B-10 


Date 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

3  Oct 

2359 

53.6 

0.00 

4  Oct 

0100 

54.5 

0.00 

0200 

54.7 

0.00 

0300 

54.7 

0.00 

0400 

54.1 

0.00 

0500 

53.6 

0.00 

0600 

52.2 

0.00 

0700 

50.9 

0.00 

0800 

54.7 

0.00 

0900 

58.6 

0.00 

1000 

60.8 

0.00 

1100 

63.0 

0.00 

1200 

64.8 

0.00 

1300 

66.0 

0.00 

1400 

66.6 

0.00 

1500 

68.0 

0.00 

1600 

68.2 

0.00 

1700 

67.6 

0.00 

1800 

62.6 

0.00 

1900 

57.4 

0.00 

2000 

55.4 

0.00 

2100 

54.9 

0.00 

2200 

54.3 

0.00 

2300 

54.0 

0.00 

2359 

53.2 

0.00 

B-l  1 
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APPENDIX  C.  SOIL  MOISTURE 


Date:  15  Sep  08 

Times:  N/A  through  1415 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

1.7 

6  to  12 

N/A 

3.4 

12  to  24 

N/A 

5.4 

24  to  36 

N/A 

3.7 

36  to  48 

N/A 

3.7 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-l 


C-2 


Date:  17  Sep  08 

Times:  1000  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

1.7 

N/A 

6  to  12 

3.1 

N/A 

12  to  24 

5.4 

N/A 

24  to  36 

3.7 

N/A 

36  to  48 

3.7 

N/A 

Blind  grid/moguls 

0  to  6 

1.6 

1.6 

6  to  12 

3.7 

3.7 

12  to  24 

3.7 

3.7 

24  to  36 

3.7 

3.7 

36  to  48 

3.7 

3.7 

C-3 


Date:  18  Sep  08 

Times:  0700  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

1.6 

1.6 

6  to  12 

3.7 

3.7 

12  to  24 

3.7 

3.7 

24  to  36 

3.7 

3.7 

36  to  48 

3.7 

3.7 

C-4 


Date:  19  Sep  08 

Times:  0700  through  1800 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.6 

5.4 

6  to  12 

8.2 

8.1 

12  to  24 

11.9 

11.8 

24  to  36 

21.4 

21.3 

36  to  48 

21.9 

21.7 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

1.5 

N/A 

6  to  12 

3.6 

N/A 

12  to  24 

3.7 

N/A 

24  to  36 

3.7 

N/A 

36  to  48 

3.7 

N/A 

C-5 


Date:  20  Sep  08 

Times:  0700  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 


Calibration  lanes 


Blind  grid/moguls 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


C-6 


Date:  22  Sep  08 

Times:  0700  through  1800 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-7 


Date:  23  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.3 

5.3 

6  to  12 

7.7 

7.6 

12  to  24 

11.3 

11.3 

24  to  36 

21.1 

21.0 

36  to  48 

21.5 

21.7 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-8 


Date:  24  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.2 

5.1 

6  to  12 

7.8 

7.7 

12  to  24 

11.2 

11.1 

24  to  36 

20.9 

20.9 

36  to  48 

21.5 

21.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-9 


Date:  25  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.0 

5.2 

6  to  12 

7.8 

7.9 

12  to  24 

11.1 

11.8 

24  to  36 

20.8 

21.6 

36  to  48 

21.5 

21.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-10 


Date:  26  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.9 

5.8 

6  to  12 

8.6 

8.8 

12  to  24 

11.9 

12.4 

24  to  36 

21.9 

21.9 

36  to  48 

22.6 

22.5 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-ll 


Date:  27  Sep  08 

Times:  0700  through  1400 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.9 

5.8 

6  to  12 

9.6 

9.7 

12  to  24 

12.8 

12.9 

24  to  36 

22.6 

22.7 

36  to  48 

23.9 

23.8 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-12 


Date:  29  Sep  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


C-13 


Date:  30  Sep  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

3.6 

3.6 

6  to  12 

3.7 

3.8 

12  to  24 

6.6 

6.5 

24  to  36 

5.3 

5.2 

36  to  48 

4.8 

4.7 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-14 


Date:  1  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.8 

6.7 

6  to  12 

10.8 

10.7 

12  to  24 

13.7 

13.6 

24  to  36 

22.5 

22.8 

36  to  48 

23.6 

23.9 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-15 


Date:  2  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.9 

6.8 

6  to  12 

10.8 

10.6 

12  to  24 

13.9 

13.8 

24  to  36 

22.7 

22.8 

36  to  48 

23.6 

23.9 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-16 


Date:  3  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

2.3 

2.3 

6  to  12 

3.8 

3.7 

12  to  24 

3.9 

3.9 

24  to  36 

5.2 

5.1 

36  to  48 

5.7 

5.9 

C-17 


Date:  4  Oct  08 

Times:  0700  through  1330 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.5 

6.5 

6  to  12 

10.8 

10.7 

12  to  24 

13.6 

13.6 

24  to  36 

22.4 

22.2 

36  to  48 

23.7 

23.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-18 


D-l 


Date 

N< 

Pe 

m 

f 

e 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

9/15/2008 

3 

CALIBRATION 

LANES 

915 

1200 

165 

9/15/2008 

3 

CALIBRATION 

LANES 

1200 

1240 

40 

9/15/2008 

3 

CALIBRATION 

LANES 

1240 

1300 

20 

9/15/2008 

3 

CALIBRATION 

LANES 

1300 

1505 

125 

9/15/2008 

3 

CALIBRATION 

LANES 

1505 

1520 

15 

9/15/2008 

3 

CALIBRATION 

LANES 

1520 

1530 

10 

9/15/2008 

3 

CALIBRATION 

LANES 

1530 

1600 

30 

9/16/2008 

3 

CALIBRATION 

LANES 

750 

900 

70 

9/16/2008 

3 

CALIBRATION 

LANES 

900 

910 

10 

9/16/2008 

3 

CALIBRATION 

LANES 

910 

1120 

130 

9/16/2008 

3 

CALIBRATION 

LANES 

1120 

1140 

20 

9/16/2008 

3 

BLIND  TEST  GRID 

1140 

1205 

25 

9/16/2008 

3 

BLIND  TEST  GRID 

1205 

1215 

10 

9/16/2008 

3 

BLIND  TEST  GRID 

1215 

1325 

70 

9/16/2008 

3 

BLIND  TEST  GRID 

1325 

1350 

25 

9/16/2008 

3 

BLIND  TEST  GRID 

1350 

1405 

15 

9/16/2008 

3 

BLIND  TEST  GRID 

1405 

1535 

90 

9/16/2008 

3 

BLIND  TEST  GRID 

1535 

1550 

15 

9/16/2008 

3 

CALIBRATION 

LANES 

1550 

1640 

50 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

INITIAL  SET-UP 

INITIAL 

MOBILIZATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

1  METER  LINE 
SPACING 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 

DOWNLOAD 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

DATA  CHECK 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 
3/4  METER  LINE 
SPACING 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

APPENDIX  D.  DAILY  ACTIVITY  LOGS 


D-2 


Date 

m 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

9/16/2008 

3 

CALIBRATION 

LANES 

1640 

1650 

10 

9/16/2008 

3 

CALIBRATION 

LANES 

1650 

1705 

15 

9/17/2008 

3 

CALIBRATION 

LANES 

750 

830 

40 

9/17/2008 

3 

CALIBRATION 

LANES 

830 

840 

10 

9/17/2008 

3 

CALIBRATION 

LANES 

840 

935 

55 

9/17/2008 

3 

CALIBRATION 

LANES 

935 

955 

20 

9/17/2008 

3 

BLIND  TEST  GRID 

955 

1220 

145 

9/17/2008 

3 

BLIND  TEST  GRID 

1220 

1235 

15 

9/17/2008 

3 

BLIND  TEST  GRID 

1235 

1310 

35 

9/17/2008 

3 

BLIND  TEST  GRID 

1310 

1330 

20 

9/17/2008 

3 

BLIND  TEST  GRID 

1330 

1515 

105 

9/17/2008 

3 

BLIND  TEST  GRID 

1515 

1535 

20 

9/17/2008 

3 

BLIND  TEST  GRID 

1535 

1630 

55 

9/17/2008 

3 

BLIND  TEST  GRID 

1630 

1645 

15 

9/17/2008 

3 

BLIND  TEST  GRID 

1645 

1710 

25 

9/18/2008 

3 

BLIND  TEST  GRID 

745 

815 

30 

9/18/2008 

3 

BLIND  TEST  GRID 

815 

830 

15 

9/18/2008 

3 

BLIND  TEST  GRID 

830 

1125 

175 

9/18/2008 

3 

BLIND  TEST  GRID 

1125 

1135 

10 

9/18/2008 

3 

BLIND  TEST  GRID 

1135 

1215 

40 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

DATA  CHECK 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

D-3 


Date 

m 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

9/18/2008 

3 

BLIND  TEST  GRID 

1215 

1235 

20 

9/18/2008 

3 

BLIND  TEST  GRID 

1235 

1245 

10 

9/18/2008 

3 

BLIND  TEST  GRID 

1245 

1435 

110 

9/18/2008 

3 

BLIND  TEST  GRID 

1435 

1450 

15 

9/18/2008 

3 

CALIBRATION 

LANES 

1450 

1520 

30 

9/18/2008 

3 

CALIBRATION 

LANES 

1520 

1530 

10 

9/18/2008 

3 

CALIBRATION 

LANES 

1530 

1550 

20 

9/19/2008 

3 

BLIND  TEST  GRID 

740 

825 

45 

9/19/2008 

3 

BLIND  TEST  GRID 

825 

835 

10 

9/19/2008 

3 

BLIND  TEST  GRID 

835 

900 

25 

9/19/2008 

3 

OPEN  FIELD 

900 

910 

10 

9/19/2008 

3 

OPEN  FIELD 

910 

1250 

220 

9/19/2008 

3 

OPEN  FIELD 

1250 

1300 

10 

9/19/2008 

3 

OPEN  FIELD 

1300 

1325 

25 

9/19/2008 

3 

OPEN  FIELD 

1325 

1335 

10 

9/19/2008 

3 

OPEN  FIELD 

1335 

1530 

115 

9/19/2008 

3 

OPEN  FIELD 

1530 

1540 

10 

9/19/2008 

3 

OPEN  FIELD 

1540 

1605 

25 

9/20/2008 

3 

OPEN  FIELD 

745 

825 

40 

9/20/2008 

3 

OPEN  FIELD 

825 

835 

10 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECT  DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA, 

INDIRECT  FIRE 
3/4  METER 
LINESPACING 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

D-4 


Date 

■ 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

9/20/2008 

3 

OPEN  FIELD 

835 

1330 

295 

9/20/2008 

3 

OPEN  FIELD 

1330 

1350 

20 

9/22/2008 

3 

OPEN  FIELD 

745 

850 

65 

9/22/2008 

3 

OPEN  FIELD 

850 

900 

10 

9/22/2008 

3 

OPEN  FIELD 

900 

1250 

230 

9/22/2008 

3 

OPEN  FIELD 

1250 

1300 

10 

9/22/2008 

3 

OPEN  FIELD 

1300 

1340 

40 

9/22/2008 

3 

OPEN  FIELD 

1340 

1420 

40 

9/22/2008 

3 

OPEN  FIELD 

1420 

1640 

140 

9/22/2008 

3 

OPEN  FIELD 

1640 

1700 

20 

9/23/2008 

3 

OPEN  FIELD 

740 

820 

40 

9/23/2008 

3 

OPEN  FIELD 

820 

840 

20 

9/23/2008 

3 

OPEN  FIELD 

840 

1255 

255 

9/23/2008 

3 

OPEN  FIELD 

1255 

1325 

30 

9/23/2008 

3 

OPEN  FIELD 

1325 

1330 

5 

9/23/2008 

3 

OPEN  FIELD 

1330 

1645 

195 

9/23/2008 

3 

OPEN  FIELD 

1645 

1650 

5 

9/23/2008 

3 

OPEN  FIELD 

1650 

1705 

15 

9/24/2008 

3 

OPEN  FIELD 

740 

825 

45 

9/24/2008 

3 

OPEN  FIELD 

825 

835 

10 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

DATA  CHECK 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

D-5 


Date 

■ 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

9/24/2008 

3 

OPEN  FIELD 

835 

1200 

205 

9/24/2008 

3 

OPEN  FIELD 

1200 

1210 

10 

9/24/2008 

3 

OPEN  FIELD 

1210 

1220 

10 

9/24/2008 

3 

OPEN  FIELD 

1220 

1250 

30 

9/24/2008 

3 

OPEN  FIELD 

1250 

1640 

230 

9/24/2008 

3 

OPEN  FIELD 

1640 

1645 

5 

9/24/2008 

3 

OPEN  FIELD 

1645 

1700 

15 

9/25/2008 

3 

OPEN  FIELD 

740 

800 

20 

9/25/2008 

3 

OPEN  FIELD 

800 

810 

10 

9/25/2008 

3 

OPEN  FIELD 

810 

1005 

5 

9/25/2008 

3 

OPEN  FIELD 

1005 

1015 

10 

9/25/2008 

3 

OPEN  FIELD 

1015 

1225 

130 

9/25/2008 

3 

OPEN  FIELD 

1225 

1230 

5 

9/25/2008 

3 

OPEN  FIELD 

1230 

1235 

5 

9/25/2008 

3 

OPEN  FIELD 

1235 

1250 

15 

9/25/2008 

3 

OPEN  FIELD 

1250 

1555 

185 

9/25/2008 

3 

OPEN  FIELD 

1555 

1605 

10 

9/25/2008 

3 

OPEN  FIELD 

1605 

1625 

20 

9/26/2008 

3 

OPEN  FIELD 

750 

820 

30 

9/26/2008 

3 

OPEN  FIELD 

820 

830 

10 

9/26/2008 

3 

OPEN  FIELD 

830 

1200 

210 

9/26/2008 

3 

OPEN  FIELD 

1200 

1225 

25 

9/26/2008 

3 

OPEN  FIELD 

1225 

1250 

25 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DOWNTIME 
DUE  TO  EQUIP 
MAINT/CHECK 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

RAINY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

RAINY 

MUDDY 

D-6 


Date 


9/26/2008 


9/26/2008 


9/26/2008 


9/26/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


9/27/2008 


09/30/08 


09/30/08 


09/30/08 


09/30/08 


10/01/08 


10/01/08 


■ 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

3 

OPEN  FIELD 

1250 

1300 

3 

OPEN  FIELD 

1300 

1600 

3 

OPEN  FIELD 

1600 

1610 

3 

OPEN  FIELD 

1610 

1625 

3 

OPEN  FIELD 

745 

820 

3 

OPEN  FIELD 

820 

830 

3 

OPEN  FIELD 

830 

1150 

3 

OPEN  FIELD 

1150 

1205 

3 

OPEN  FIELD 

1205 

1235 

3 

OPEN  FIELD 

1235 

1240 

3 

OPEN  FIELD 

1240 

1420 

3 

OPEN  FIELD 

1420 

1425 

3 

OPEN  FIELD 

1425 

1435 

3 

OPEN  FIELD 

1430 

1610 

3 

OPEN  FIELD 

1610 

1630 

3 

OPEN  FIELD 

1630 

1640 

3 

OPEN  FIELD 

1640 

1700 

3 

OPEN  FIELD 

750 

820 

3 

OPEN  FIELD 

820 

825 

3 

OPEN  FIELD 

825 

1200 

3 

OPEN  FIELD 

1200 

1210 

Operational 

Status 

Operational 
Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 
DATA  DIRECT 
FIRE 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 
DATA  DIRECT 
FIRE 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

BREAK/LUNCH 

BREAK/LUNCH 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 
DATA  DIRECT 
FIRE 

GPS 

LINEAR 

RAINY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

RAINY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 
DATA  DIRECT 
FIRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

COLLECTING 
DATA  DIRECT 
FIRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

(Page  D-8  Blank) 


Date 

No.  of 
People 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

10/01/08 

3 

OPEN  FIELD 

1210 

1245 

35 

10/01/08 

3 

OPEN  FIELD 

1245 

1505 

140 

10/01/08 

3 

OPEN  FIELD 

1505 

1510 

5 

10/01/08 

3 

OPEN  FIELD 

1510 

1550 

40 

10/02/08 

3 

OPEN  FIELD 

750 

805 

15 

10/02/08 

3 

OPEN  FIELD 

805 

815 

10 

10/02/08 

3 

OPEN  FIELD 

815 

1310 

295 

10/02/08 

3 

OPEN  FIELD 

1310 

1330 

20 

10/02/08 

3 

OPEN  FIELD 

1330 

1340 

10 

10/02/08 

3 

OPEN  FIELD 

1340 

1635 

175 

10/02/08 

3 

OPEN  FIELD 

1635 

1645 

10 

10/02/08 

3 

OPEN  FIELD 

1645 

1710 

25 

10/04/08 

3 

OPEN  FIELD 

750 

810 

815 

10/04/08 

3 

OPEN  FIELD 

810 

820 

10 

10/04/08 

3 

OPEN  FIELD 

820 

1240 

260 

10/04/08 

3 

OPEN  FIELD 

1240 

1250 

10 

10/04/08 

3 

OPEN  FIELD 

1250 

1345 

55 

Operational 
_ Status _ 

BREAK/LUNCH 

COLLECTING 

DATA 

CALIBRATION 

DAILY  START, 
STOP 

DAILY  START, 
STOP 

CALIBRATION 

COLLECTING 

DATA 

BREAK/LUNCH 

CALIBRATION 

COLLECTING 

DATA 

CALIBRATION 

DAILY  START. 
STOP 

DAILY  START. 
STOP 

CALIBRATION 

COLLECTING 

DATA 

CALIBRATION 

DEMOBILIZATI 

ON 


Opel 

St 

Con 

ational 

itus  - 

iments 

Track 

Method 

Pattern 

Field  Conditions 

BREA! 

C/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLL 

DATA 

F 

ECTING 

DIRECT 

IRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

EQUI 

BREA 

PMENT 

KDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

SE 

EQUI 

TUP 

PMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLL 

DATA 

F 

ECTING 

DIRECT 

IRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

BREA! 

C/LUNCH 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLL 

DATA 

F 

ECTING 

DIRECT 

IRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

EQUI 

BREA 

PMENT 

KDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

SE 

EQUI 

TUP 

PMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLL 

DATA 

F 

ECTING 

DIRECT 

IRE 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIB 

RATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

DEMO 

BILIZATI 

DN 

GPS 

LINEAR 

SUNNY 

MUDDY 
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APPENDIX  F.  ABBREVIATIONS 


AOL 

APG 

ATC 

BAH 

BAR 

BUD 

DAQ 

DMM 

DPRT 

EM 

EM/MAG 

EMI 

EQT 

EQT 

ERDC 

ESTCP 

FOM 

GPS 

GS 

GT 

GX 

HEAT 

IDA 

JPG 

MM 

NAVSEA 

NS 

POC 

QA 

QC 

ROC 

RTK 

S 

SERDP 

SNR 

TEM 

USAEC 

UXO 

YPG 


advanced  ordnance  locator 
Aberdeen  Proving  Ground 
U.S.  Army  Aberdeen  Test  Center 
Booz  Allen  Hamilton 
background  alann  rate 
Berkeley  UXO  discriminator 
data  acquisition 
discarded  military  munitions 
Duke  Pattern  Recognition  Toolbox 
electromagnetic 
electromagnetic/magnetometer 
electromagnetic  induction 

Army  Environmental  Quality  Technology  Program 
Environmental  Quality  Technology  Program 

U.S.  Army  Corps  of  Engineers  Engineering  Research  and  Development  Center 

Environmental  Security  Technology  Certification  Program 

figure  of  merit 

Global  Positioning  System 

Geosoft  Script 

ground  truth 

Geosoft  Executable 

high-explosive  antitank 

Institute  for  Defense  Analysis 

Jefferson  Proving  Ground 

military  munitions 

Naval  Sea  Systems  Command 

nonstandard  munition 

point  of  contact 

quality  assurance 

quality  control 

receiver-operating  characteristic 
real-time  kinematic 
standard  munition 

Strategic  Environmental  Research  and  Development  Program 

signal-to-noise  ratio 

time-gated  electromagnetic 

U.S.  Army  Environmental  Command 

unexploded  ordnance 

U.S.  Army  Yuma  Proving  Ground 
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Aberdeen  Proving  Ground,  MD  21010-5401 
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